After what Rousso describes as a childhood full of ups and downs and other children asking why she was "crippled," Rousso grew up to pursue higher education despite what some saw as barriers—her disability and her gender. She earned a degree from Brandeis University in economics in 1968, which landed her a job in Washington D.C. in the Office of Economic Opportunity and also exposed her to the women's rights movement. It was while living in D.C. that Rousso became involved in feminist activism. Rousso said in an interview that when she started working in feminist issues, she realized that "this self-loathing about my body and about my womanhood is not just a disability issue, it is a women's issue." Her first taste of activism fueled her to go on to get two masters degrees, one in education from Boston University and the second in social work from New York University. She had hoped to study psychotherapy but was rejected from a program because of her disability. Instead of being discouraged, this action further inspired her life work at the intersection of disability and women's rights.