He’s So Vain: Carly Simon and The Wannabe Madoff

Ken Starr bilked the rich and famous and, now, incredulous

On a recent Thursday morning, Internal Revenue Service agents walked up to a shiny new building on E. 74th St. in New York. Bernie Madoff’s younger son has a $4.4 million condo on the fifth floor, but they were looking for his neighbor. “Tell them I’m not here,” the money manager Kenneth I. Starr said to the doorman from the $7.6 million apartment he bought a month and a half ago. “Only my wife is here.”

An hour later, the agents were about to use a master key when Ms. Starr, a former erotic dancer, opened the door: “He’s,” she whispered, “upstairs.” The 66-year-old financial adviser, whose clients have included Henry Kissinger, Caroline Kennedy, Bunny Mellon and Martin Scorsese, was found hiding in a closet, his shoes sticking out.

Eight days later, Carly Simon was lying in bed in Martha’s Vineyard. Wearing a white nightgown, she answered the phone. “I’m fine,” she said. “I’m not fine, what am I saying?” For one thing, she’s been ill: She thinks she was bitten by a tick when her horse was put down. For another, the singer says she has lost upward of $15 million to Starr, who is being held without bail on charges that he was running a Ponzi-like scheme.

In a complaint that accuses him of fraud and money laundering, prosecutors say Starr (no relation to the Clinton foe) bought his new apartment with $5.75 million taken from Mellon, a 99-year-old widow, and $1 million from Uma Thurman. He repaid the actress when she went up to his office to confront him two weeks later, using money taken from his client Jim Wiatt, the former William Morris chairman.

Simon, whose father co-founded Simon & Schuster, thinks her own money is gone because of Starr’s negligence and fraud, which she wasn’t sharp enough to catch in time. “You’re talking to somebody I’d rather not be,” she said. “I don’t like to admit how little I knew about anything financial.”

“She leaves it to people that she trusts,” said her sister, Joanna, a Manhattan real estate broker, and the late Walter Cronkite’s companion. “And it turns out a lot of those people were untrustworthy. She’s a real artist, and she needs people to look after her.” Cronkite, too, she said, was a client of Starr’s, though he got out early.

If an enormous lesson of the financial crisis was that most of Manhattan was cheating, one moral to the story of the sprawling Madoff disaster and now the cinematic Starr charges is that most everybody is being cheated, even the artistes and statesmen and dazzlingly aristocratic widows.

In the mid-’90s, the singer and her ex-husband, James Taylor, happened to be using the same money manager. “I felt that I was second best, and I wasn’t getting the kind of attention that a pasha like myself would want,” she said. “James just always was much more of a worker bee than I, and therefore it reflected in the kind of attention from a business manager that one would expect.” She heard about Ken Starr from three different people at once. “All very successful and moneyed,” she said. “They were all very importantly involved with the theater and television.”

Still, when she went to meet the adviser, she brought along a friend, billionaire Dirk Ziff, who didn’t raise any red flags, though nor did he want to use Starr himself.

As Simon remembers it, Starr told the singer he knew her work—her songs include “You’re So Vain,” “Nobody Does It Better” and “That’s the Way I’ve Always Heard It Should Be.” Starr said that even though her wealth fell below his threshold, he’d take her on as a client and piggyback her account onto Blackstone private-equity deity Pete Peterson’s. She didn’t understand how that would work, but she liked the sound of it. “I thought, ‘Wow. I struck gold.’” He guaranteed a return of 28 percent. “Of course, in retrospect, one sees extremely differently, but I thought he was the type of power that I hadn’t come across.”

Up until recently, Starr had rarely been photographed without a charming and important person by his side. Around town, he could be seen sipping Diet Coke at the Four Seasons at lunch, or maybe chatting to other guests at Bank of America’s ritzy private client event about his time at the glorious Allen & Co. retreat.

Then there are odder images, like the shots of him with his tattooed and buxom wife, his third, next to bodybuilders at their American Gladiator–themed private cocktail party. But even at the pole-dancing parties she threw, there are snapshots of Starr with David Blaine or Scarface producer Marty Bregman. Al Pacino was a client, too.

So were Mike Nichols, Annie Leibovitz, Neil Simon and Barbara Walters. And during a recording of a conversation with the Los Angeles private investigator Anthony Pellicano that was leaked during his trial, Starr said the billionaire Mort Zuckerman had been a client and friend, too. “That’s not my recollection,” Zuckerman, also a Madoff victim, said this week. “I’m going to tell you something: For years I have been asked by various friends in the entertainment world, and I have advised them all to stay away from this guy.”

At first, the singer and the adviser got along fine. “Smooth sailing,” Simon said. “Things looked as though they were moderately improving for the first four or five years.” She didn’t talk to Starr very often, though she sometimes saw him up on Martha’s Vineyard.

When the fall started, it was too slow to bother her. There were sporadic warnings from friends, and there were official complaints in court from clients like Sly Stallone, Wesley Snipes and Joan Stanton, the widow of a Volkswagen mogul and the radio voice of Lois Lane. Simon got a new assistant, who saw she was being billed for odd storage units—and, somehow, James Taylor’s health insurance. There was a tax mishap. There was a piece of paper sent to Martha’s Vineyard that would have sold off her favorite investment had she signed. There was an enormous line of credit established.

But it was not a crisis. “My life was so filled with so many other problems,” she said. There were health issues to worry about, and her children, plus her relationship with Starbucks, which she eventually sued over an album. “‘We’ve set it up,’ said Ken Starr to me, ‘so that you will not have to worry for the rest of your life.’”

A few months after the collapse of Lehman Brothers, the singer got a call from a woman at the Starr office. She wanted whatever money that was in Simon’s Martha’s Vineyard account transferred to New York, because there suddenly wasn’t enough money to pay staff members, like the assistant. “That’s when I began to call Ken,” she said.

“Why would you want to get in touch with him?” someone at his office said. In January, as she tells it, she waited an hour to speak to him. “I said, ‘Ken, why don’t I have any money?’”

“What do you mean?” he said. “Everything’s fine.”

She left him, wiped out. Beyond the question of negligence, she and her attorney, Marty Singer, say she may be a victim of the Ponzi scheme he’s been accused of. Last year, The Times called him the “go-to guy for star-crossed stars.” “We are at the stage,” he said this week, “when we could be suing.”

“She claimed things, that he allowed her to spend too much money on her house, stuff like that,” said the attorney Bert Fields, who has done work for Starr for years, and helped him with Simon’s complaints earlier this year. “That’s the kind of thing my wife might accuse me of.”

“Oh, bullshit,” she said. “I think you spend too much on your investments based on what your money manager tells you to spend.”

“I would be reluctant to file the suit,” Fields said. “But listen, it costs $65, I think, to file.”

Starr was arrested with the former president of the New York City Council, Andrew Stein, who was charged with lying to federal officers about a shell company that was used for Starr’s fraud. Attorney Ed Hayes, an inspiration for Tom Wolfe’s The Bonfire of the Vanities, met with Starr in jail. “He didn’t seem cocky in the slightest; he seemed very disheartened and upset,” Hayes said.

Hayes was only involved for a few days. “In the beginning, he said, ‘I’m going to have the money,’ and then he didn’t come up with it.” When Simon heard about the agents and the arrest and the jailing, she said she felt sorry that she hadn’t left Starr earlier. “There was one friend of mine that said, ‘I just feel I’m too old to deal with this. He takes me out to lunch when I’m in town!’ So there was one friend of mine who just wasn’t going to do anything about it.”

Her West Village duplex has been on the market for years, but she still has her house on Martha’s Vineyard. There, waiting for a Lyme disease test early this week, she was in bed again in the late afternoon, but optimistic. “I’m a believer,” she said. “So I’m a believer that the next great thing could happen.”

Suggested Next Read

Lakers worth the cost despite hefty price

Going for Broke

Lakers worth the cost despite hefty price

By Matt Jacob

Siegfried & Roy. Beavis & Butt-head. Mark McGwire and a syringe. O.J. and handcuffs. Pizza, beer and my mouth. Let’s face it, some things are meant to go together—which brings me to the NBA Finals. For the 12th time in history—and second time in three years—the Lakers and Celtics will clash for the NBA championship. Although the league’s executives and advertisers were deprived once again of a LeBron-Kobe matchup, they got the next best thing.