The name was carefully chosen to help with the branding—it helps to have a familiar-sounding name that can easily be turned into a verb, according to Ballmer, who also reportedly had fun repeatedly saying "bing!". (The timing of the announcement was less carefully chosen: Bing.com won't be live until June 3.) Bing is a pretty direct reference to the success of the name Google, which is Microsoft's target, after all. It's also a short URL, and easy to remember so people can talk about it... and it most definitely wasn't named for Bing Crosby.
As far as the technology goes, at the All Things D demo, Ballmer was keen to show that Bing will be more relevant than existing search engines by prioritizing certain kind of results at the top of the list. For example, searching a company results in its customer service numbers being highly placed on the list so that you don't have to scour corporate websites for one. Searching for a place results in a small map, a list of attractions and current weather conditions. That real-time-ness was also demonstrated with live flight info and stock prices.
Ballmer also showed how you can get previews of the linked search results by simply hovering over them (an existing tech widely used on the Web already), and that the engine tries to proactively help your searches by adding quick-tabs on a side bar that connect to ongoing queries. For example, if you've searched for a product, Bing might pop up a tab with a product review search query already populated with results.
Bing is genuinely designed to suggest and organize stuff for you in an intelligent way. This is so key that Microsoft's even given the tech a name, "The Decision Engine," and it's main purpose is to try to lead you to making a choice. That's a great contrast to Google, which simply spews up a list of linguistically-similar search results, hence leaving the decisions totally up to you. You could, of course, use some of Google's advanced new features to refine the data down to help you make a better decision—or use Wolfram Alpha's vast fact database to do the same. But that's still basically all your effort to hunt through and prioritize the results.
All in all Bing does sound pretty powerful, even if the very best results were chosen for this particular demonstration.
Ballmer, who was almost as focused on the branding of Bing as on its technology, let slip one more important detail: Microsoft will be arranging for PCs to come pre-configured with Bing as the default search engine. This will help to promote it, and get it into the public consciousness. And it also sounds like another potential anti-trust case in the making, along the lines of the pre-configured Internet Explorer issue.