Today, Mary-Kate and Ashley Olsen are award-winning fashion designers responsible for two brands, The Row and Elizabeth & James. But before they from their matchy-matchy Full House days to their current status as style icons, there was one woman responsible for their wardrobes: Judy Swartz.

When they started working with Schwartz, the Olsen twins were in the midst of pumping out their string of straight-to-VHS successes (Billboard Dad, Passport to Paris, Winning London, and the like) as well as dipping their toes into fashion for the first time with their Walmart clothing line—and Swartz had a heavy hand in it all. "Anything that had to do with fashion, I was working on," she revealed to Refinery 29.

As such, Swartz had a significant role in helping the coordination-loving Olsen twins define and hone their style. While other tweens their age were shopping at Limited Too and Abercrombie & Fitch, the Olsen twins were wearing looks inspired by high-fashion designers and famous artists. "I would emulate high-end designers: I'd look at a couple of Miu Miu pieces and a few from Dolce & Gabbana; one collection might have been Chanel-inspired," Swartz said of her inspiration in dressing and designing for the girls.

The same principle carried over to styling their various straight-to-video films. "Holiday in the Sun was very summery with a beach vibe, so I used pieces from Pucci and Calypso," Swartz recalled. Others, like Our Lips Are Sealed, featured pieces inspired by artists such as Piet Mondrian.

Once the Olsens were older, they developed their own favorite designers and brands to turn to. "You know what we used to love? Earl Jeans," Swartz says. "And then, when Earl Jeans made leather pants, I bought those and cut them down. We also used a lot of Theory. I bought lots of Prada, Miu Miu, and some Gucci, too."

Although Swartz hasn't worked with the Olsens in some time, she's claiming credit for one of their current signature looks: layering. "Well, the layers came from me. It's how I dress," she said. "I'm not surprised that layering moved forward with [Mary-Kate and Ashley's] aesthetics, because it was such a big part of their style growing up."

Related: Getting candid with fashion's top designers: