Your agreement complicates things, only a lawyer can advise you of your legal position, and no one can tell what the outcome of your friendship will be. That said, if it weren't but for the agreement, I would personally sleep perfectly well if I were in your position -- people sit around and spitball ideas all the time, a telltale sign of this is when the ideas are discussed in terms not much deeper than "do X like Y game, but also with W from Z book." I mean, people mention vague premises they want to explore, or very generalized game mechanics all the time -- and since those are not "designs" its well and truly bullshit for them to come back and accuse anyone of stealing their design when someone else has gone off on their own and put their own spin to it.

Now, if they guy provides value in some capacity -- design, business, etc. -- and you want him onboard for the future, just tell him that there's a place for him in a future company or whatever, but that the 50-50 notion just doesn't seem fair given the disparity of what you've put in -- keep in mind, however, that the first step is to fairly acknowlege the value he's provided not just for this game in specific, but in all things under your agreement that has brought you to where you are now, because your upcoming project didn't simply spring forth from its ideation, but also from all of your collaborative works before. Likewise, weigh the actual likely financial gain this might lead you to going 50-50 on $100 million is a very different story than going 50-50 on $1000, and even if it may not be 'fair' it might not be worth taking the lion's share of the latter if it means damaging a friendship.

Finally evaluate what he has done and what role he will continue to play going forward, if any. 50-50 doesn't seem like a fair split for what he seems to do for you. Likewise, you'll want to keep some money in the business if you're serious, and that should be figured in before either of you take a penny for yourselves. This is a valuable lesson in thinking through your agreements before letting them take root.

A verbal agreement is purely informal in most jurisdictions unless you've signed something. Legally they hold very little water, just thinking back on this from when a friend and I worked together on a business idea. However, friendships are worth more than money (at least to me) so i'd be inclined to honour the agreement regardless making clear that your friendship is more important than falling out over finances. Business has the potential to destroy friendships and everything should be made clear and in writing whenever money is involved. If in doubt in the future, speak to a lawyer...