Rick Stein reveals why he split with wife of 31 years after fling with publicity manager TV star, 66, told The Times' Magazine he 'kind of fell out of love' with Jill They divorced after five-year affair with publicity manager Sarah emerged He has opened up about the strain of his affair - both for him and first wife New book says he suffers from crippling self-doubt and wants to be liked Millionaire chef Rick Stein has blamed the stress of the restaurant industry for making him 'fall out of love' with his first wife and have a five-year affair with a pretty blonde 20 years his junior. His first wife Jill still helps run the 66-year-old's empire - but once came to blows with her husband's publicist Sarah Burns, slapping the pair of them outside one of his restaurants in Padstow, Cornwall. He has now been married to Sarah for two years. In a revealing interview today he lays the blame on himself and his 'bloody demanding' restaurants. 'We kind of fell out of love': Rick Stein with his first wife Jill, left, and his new bride, his publicist Sarah Burns Rick Stein is in fact full of insecurities, regularly suffers crises of confidence and has dreams about working in a kitchen and being unable to get the food out, he has said as he releases a new memoir He told The Times' Magazine: 'The business was eating us alive. What happened had nothing to do with Jill. It's more to do with the nature of the restaurant business - it's just so bloody demanding. 'It takes over your life and our lives didn't exist outside it. The more successful we got, we had to keep the restaurant going all year and then we didn't go away on our holidays. We didn't have a life outside the business and we kind of fell out of love. 'I'd never [been unfaithful] before and I certainly won't do it again, that's for sure. I'm just so happy now. Nobody looks back at those times with anything but slight gloom, right? Obviously, it's not great for the person who has been left, but it's not great for you either.' He and Jill had been married for more than 20 years when in 1997, then 50, the chef began an affair with his pretty blonde Australian publicist Sarah Burns, two decades his junior. Despite Sarah being on the other side of the world he kept the affair secret from Jill for five years - and she only found out when she discovered a heap of international phone bills. Rick and Jill opened the Seafood Restaurant on the site of their former nightclub in Padstow, Cornwall The chef's empire - including 19 cookery books and several restaurants - is worth an estimated £35 million But the father-of-three did not leave Jill straight away - he dithered, a victim of what he describes as his own crippling insecurity. He and Sarah, who was also married with two children when they met, fought and he returned to Jill, begging to be taken back. She refused and he went back to Sarah. They married two years ago. Rick and Jill at the opening of their fish and chip shop in Padstow in 2004 - she still helps run the firm He told the Daily Mail earlier this year: "I know it's not possible, but I just want everyone to like me. But it's not possible because you can't help do things that p*** people off, so you might as well not bother. 'Sarah's brilliant - she's so bossy, she's really fierce, but I like that. She takes me in hand. The chef revealed to The Times' Magazine that when he is alone in Padstow he will visit his own pub three times a week with old friends - though only for an hour at a time, he insisted. His claim on Padstow is not a random one - it is more than 30 years since he opened his nightclub with Jill, which he said had fights every weekend and 'the paving slabs were very absorbent, so it was really difficult to get the blood off.' He holds no ill will to his first wife despite their long history, including the infamous slapping in 2008 outside one of his four restaurants in Padstow, Cornwall (known to locals as Padstein). In fact, the pair still attend board meetings together and swap e-mails about his empire, which includes 19 cookery books, an estimated £35 million and a head chef's post in a restaurant in New South Wales, Australia. 'I certainly would have preferred if that hadn't happened, I can tell you,' he told the Mail earlier this year. I remember at the time saying to Jill: "That was a very silly thing to do." But emotions were very high. It was bloody tough. It was horrid. Thank goodness, time moves on. Twice now, both Sarah and Jill have been in the same restaurant at the same time and it's been fine and nobody's been slapped!' 'We'd both put so much effort into building up the business that it was just too important to us not to [continue running it together]. Rick Stein's autobiography Under a Mackerel Sky (Ebury Press, £20) is out now. Rick Stein is also a TV star - but when alone in Padstow he visits his own pub three times a week, he said