Jayne Ozanne quit the panel, along with two other advisers, claiming delays in banning conversion therapy.
She told the BBC Ms Badenoch did not understand the LGBT+ community and should stand aside for someone who “had more heart for the work”.
The government vowed to produce a plan to “end” conversion therapy “shortly”.
Conversion Therapy is the pseudoscientific practice of trying to change an individual’s sexual orientation from homosexual or bisexual to heterosexual using psychological, physical, or spiritual interventions. There is no reliable evidence that sexual orientation can be changed, and medical institutions warn that conversion therapy practices are ineffective and potentially harmful.
Campaigners want legislation banning the controversial practice, which aims to change sexual orientation or gender identity.
Downing Street said it was considering “both legislative and non-legislative options” to “bring an end to” it. However, asked if that meant it could end up with measures short of a ban, a spokesman refused to go further.
In July last year, the prime minister said the practice was “absolutely abhorrent” and said plans to ban it would be brought forward.
Debate in Parliament
In a debate in Parliament on Monday, Ms Badenoch said conversion therapy had “no place in a civilised society”, but added that “robust” laws were already in place to deal with the most troubling examples, involving violence and sexual assault.”
She said the government did not “intend to stop those who wish to seek spiritual counselling as they explore their sexual orientation.”
But she added: “There will also be cases where a line is crossed, where someone is actively seeking to change another’s sexual orientation.”
“The government will be exercising great care when considering what constitutes conversion therapy, what does not and therefore how the government should intervene,” Ms Badenoch said.
An awkward issue for Prime Minister Boris Johnson
There have been rumblings for a while about a growing unease among some panel members which are now playing out in a very public way.
This advisory group was first set up under Theresa May but has not met for some time – I am told at least six months.
And now we have heard claims that, even when it did meet, panelists did not necessarily feel listened to.
If the group does appear increasingly defunct (and we will have to see what happens) it may be seen, by some, as symbolic of a fading commitment to LGBT+ equality under this administration.
Ministers would reject that and, during a debate on Monday, Kemi Badenoch said “The government had a “proud record” of championing equal opportunity.”
But it was her response overall that seems to have sparked these resignations.
Among concerns was her refusal to commit to a legislative “ban” on so-called conversion therapy, promising instead only to “end” the practice.
The equalities minister described the legal landscape as “complex”. But it is an awkward one for Boris Johnson who did, last summer, use the word “ban” during a TV interview.
When she resigned from her advisory role on Monday, Jayne Ozanne accused ministers of creating a “hostile environment” for LGBT+ people.
Two others who then also quit the panel, James Morton, and Ellen Murray, said they also had concerns about the government’s approach to conversion therapy and trans rights.
Labour Leader Sir Keir Starmer said the government “has a blind” spot in the issue.
“The PM needs to address what is obviously a problem in his government,” he said at his party’s local election campaign launch.
Asked if he would ban conversion therapy if he were prime minister, Sir Keir replied: “Yes, absolutely.”
The LGBT Advisory Council was set up in 2019, with the term of current panel members set to end on 31 March.