“Whatever you do, don’t burn that bridge,” a coworker gently reminded me after I revealed I was planning on resigning that morning. “You don’t want to get into the details of what really happened and why you are leaving.”

But over the course of my career, I have learned that the stigma of “burning bridges” often holds people back from speaking out. And if we want to build a better world of work, we need to encourage people to put aside their fears.

For 13 months, I had reported into a toxic bully in the beauty industry. This former boss would openly mock coworkers and make fun of “their accent.” She would tell us we were stupid and slow to our face. She would yell, shout, and curse in meetings. She would monitor when we came in, when and how long we took lunch breaks, and point out our lack of work ethic when we left “early” at 8 p.m. She complained that my Indian lunch was too smelly, told me that my clothing was “basic” and I lacked fashion sense, and asked me why “you people” always have “hard to pronounce” names. She even hurled one of her Chanel shoes at a colleague.

She ruled with fear. And I was early in my career, too scared to speak out while I was working for her. I would shudder as I watched her walk around the floor, terrorizing other coworkers before she made her way over to me. I thought resigning would be my moment to speak my truth, spill it all, and loudly proclaim to everyone what most everyone already knew: This toxic bully was the real reason I was leaving. And I couldn’t. I didn’t. In the end, I told HR I was leaving “for a better opportunity.”

Because for all of my career, I had been taught and trained to do otherwise. Stay professional, take the high road, don’t say anything that could be perceived as slightly negative or ruin relationships. Because we don’t want to be labeled as a troublemaker, a pot stirrer, or acting like a spoiled toddler who didn’t get their way. So, I understood that whatever you do, you never burn that bridge.

The term “burning bridges” has origins dating back to the 1800s. It refers to a popular and well known military practice, also said to have been used in Roman warfare. The idea was to burn bridges over rivers to stop the enemy from advancing any further. Or to burn bridges to stop enemies from escaping once they entered a territory. And finally, burning bridges could cut off the enemy’s supply chain. Burning bridges meant there was no way to retreat, there was no going back, and there was no other option but to move forward.

Unfortunately, we have weaponized the term “burning bridges” in our workplaces. As leaders, we need to reframe the conversation. It’s not about burning bridges, detonating relationships, and blowing up our work experiences. It’s about speaking up, sharing the truth, and parting ways, drawing lines and boundaries for those we no longer want to keep in touch with who have caused us or others harm. And, ultimately we may need to distance ourselves and protect our leadership brands from other toxic individuals and places.

Here are three moments in your career when it may make sense not to burn the bridge, but rather create the professional space and distance required:

1. IF SOMETHING UNETHICAL HAS OCCURRED

“When your company’s integrity is at stake, or your own personal reputation, you may have to immediately distance yourself from the situation,” recommends Minsun Byun Kevers, cofounder of Unbound Labs. Byun Kevers’ work is focused on building user-owned marketplaces, helping to facilitate matching companies with hard to find experts, particularly in the people and culture space. In her role, she is focused on helping to nurture and grow relationships in her marketplaces.

“The most important thing is that each party is treated with respect and that the consultants are paid fairly and equitably by companies,” she says. “If a consultant or a company is in breach of contract, doesn’t deliver on the terms that were agreed to, and isn’t open to amicably resolving the situation and becomes uncooperative and potentially hostile, it’s ultimately not someone we want to be part of our community.”

As Byun Kevers suggests, unethical and unlawful behavior can include being in breach of an agreed upon and signed contract. It can also include inflating business expenditures, embezzling funds, and violating conflict of interest or anti-corruption policies. It can also involve bullying and sexual harassment. When something unethical has occurred, reporting it to the appropriate company channels, seeking legal guidance and counsel, and distancing yourself from the individual or parties can be the best course of action—even if some may see it as “burning bridges.”

2. IF YOU HAVE BEEN NEGATIVELY IMPACTED BY SOMEONE ELSE’S ACTIONS.

“Research shows that confidence ebbs and flows at work. When someone else’s behavior negatively impacts you directly, it can chip away at your confidence and cause you to doubt yourself and your capabilities,” says Christy DeSantis, founder and chief confidence officer at Fiducia Coaching.

DeSantis is a certified coach who aims to spark confidence in women in business by helping her clients understand that they need both inner confidence and supportive work environments that foster it. “I’ve seen clients’ self-worth shattered because they were mistreated repeatedly in the workplace,” she adds. “In that case, finding a company that values you, and does not tolerate a hostile environment, is critical. That means taking those toxic relationships off the table, distancing yourself, and moving on.”

This could mean cutting yourself off from former bosses or colleagues, or even no longer associating yourself with your former workplace. Blocking people in social media, no longer accepting their phone calls, or ignoring requests to be a reference or refer them to a job at your new company is not about you being unprofessional and burning bridges. It’s about setting and maintaining boundaries for yourself.

3. IF OTHERS AROUND YOU HAVE BEEN NEGATIVELY IMPACTED BY SOMEONE ELSE’S ACTIONS

Finally, if others around you have been negatively impacted, you may decide you can no longer be associated with the individual causing harm in the workplace. In my case, after I resigned from my company, I cut off contact with my former boss despite her many attempts to contact me. One of my colleagues who was not impacted by her also cut off contact once they left the organization.

They watched what she did to me and others during that period of time and saw that leadership didn’t intervene despite repeated incidents being reported. Years later, this former colleague still refuses to respond to her repeated attempts to reconnect and network. They have gone on to build an incredibly successful career; they also refuse to be in touch with the then-division president who allowed this behavior to continue under their leadership

“Why would I want my leadership brand to be associated with someone who was so toxic? And with others who allowed that toxicity to seep through the organization?” says Christian*, my former colleague. “I don’t want anyone ever thinking I would refer her for a job or would want to work with her again or her boss. I hope has gotten the help she needs.”

Ultimately, the cost of not allowing people to take distance from someone they would like space from can cause damage to our mental health and personal leadership brands. Shaming people from asserting healthy boundaries under the advice to “not burn bridges” can stop us from healing and regaining our confidence. It can also prevent us from moving forward and growing in our professional careers. Let’s stop weaponizing the term “burning bridges.” Let’s normalize walking away from people, places, and spaces that no longer serve us.

*Name has been changed to provide anonymity.