RIM’s WES 2008 event wasn’t just about the shiny devices (though the Bold is very impressive). It’s also about the nuts and bolts behind RIM’s platform, and about how you actually run your own Blackberry service. With plenty of RIM staff on hand, and giving presentations on everything from web design for Blackberry browsers to configuring RIM’s tools to work with the latest versions of Lotus Domino, there’s a lot of material to help you through your working day.

Blackberry Enterprise Server is a hefty piece of software, and RIM’s own network administrators came down to Orlando to show how they run their service. Not surprisingly they have one of the most complex BES infrastructures around, supporting all three mail server platforms (Exchange, Domino, Groupwise) and with three distinct user classes: bleeding edge Alpha users, advanced Beta users, and general everyday Production users.

With a complex environment like that, spread across the world, what’s the secret sauce? It turns out to be their domain database strategy, which is regionalised and segregated (by mail platform and by user class). Even so, RIM aim to have as few databases as possible, with the intention of keeping management simple.

The team gave out some numbers. They currently support 9300 Blackberrys, of several different generations. There are 42 BES servers in the company, supporting all the different combinations of geography, user type, and mail server (of which there are more than 55). All that’s handled with 9 domain databases - running on centralised high availability SQL Server systems with remote connections from RIM’s global BES network.

If you’re worried about your BES performance you can take a tip from RIM, which uses a mix of virtual and physical BES systems. They use 2GB of RAM for Exchange nodes and 4GB of RAM for Lotus (as BES runs on the same server as Domino). Best results come from placing BES local to the mail server it’s using as a message source. The servers are also connected to the local PABX systems, as part of RIM’s new voice service.

So how’s it all managed? Users are supported with a service desk and with self-service. Self-service is increasingly important, and using RIM’s web-desktop manager rather than the standalone desktop tools they can set their own activation passwords and upgrade devices without having to call on the help desk and server administrators.

Server administration is handled by the corporate database team and a team of BES administrators, with the aim of tracking the code people are using on any given day. Systems are monitored using familiar enterprise management tools, including HP Open View and Microsoft System Center. A reporting database handles configuration queries, while custom scripts and the Blackberry Enterprise Resource Kit handle log analysis (though there are always ongoing evaluations of alternate tools).

It’s important for RIM to have good management tools and practices, as it’s using several different versions of the BES code - the current release, the next service pack, and the next generation release. At any time 1000 users are early adopters, already on the next generation Blackberry Enterprise Server.

So what are the five key tips from RIM’s own administrators for a successful BES implementation?

1. Keep your messaging environment healthy
2. Protect the domain databases
3. Ensure adequate server resources have been provisioned
4. Remove orphaned/unused accounts from your mail servers
5. Always document custom configurations (and also save logs)

– Simon

We spent some of Friday powering up and down the 101, meeting folk at both Serena and Yahoo!.

There was a common theme to the meetings - the power of open APIs to power the next generation of consumer and business applications. It’s well documented, discoverable, APIs let us build the complex mashups of services and processes.

Discoverable is the key word. We need to be able to automate API access in our development tools, whether they’re GUIs like Serena’s new business mash-up tool, or whether they’re JavaScript code in my web development tool of choice, Aptana.

Yahoo! has been working on API-level tooling for sometime now, and a recurring theme of our lunch conversation was summed up in a question from a developer evangelist: “What APIs can we offer you next?” It was a question that made me think, as Yahoo!’s APIs have been at the heart of the web applications I’ve been writing recently. It’s Yahoo! Local Search that geocodes my postcodes for me, and Yahoo! Pipes that converts any web service into a simple JSON operator I can use in JavaScript to build cross-service mashups that down fall foul of the browser security model.

The latest tool to come out of Yahoo!’s research teams is Fire Eagle, a universal location broker. Tell Fire Eagle where you are, and you can share your location with applications that you’ve given access rights. The Fire Eagle API is designed to handle location information (along with the details of the providing service, so you know how accurate the information is), and authentication (making sure the right person gets the right level of detail).

APIs like these are a key to delivering on two key visions: cloud computing and SOA. When you’re using Fire Eagl, you’re subscribing to a service, either as an information provider or an information consumer. You’re also taking advantage of the infrastructure Yahoo!’s built, using compute resources in the cloud to manage your individual location information.

Analyst James Governor came up with an interesting list of signs that something isn’t cloud computing. At number 5 on the list was this: If there is no API… its not a cloud.

I can’t disagree with that, at all.

–Simon

The UK government wants ISPs to control the traffic of illegal file sharing through their networks. If they don’t, the government is planning to introduce legislation to ensure they do. I’m left stuck with an image of King Canute trying to hold back the tide. Failed business models don’t need to be propped up with legislation…

So why isn’t it going to work?

First, some basic numbers. The BBC article on the government proposals suggests that there are 6 million broadband accounts downloading illegal files every year. That’s a hefty proportion of the UK’s 15 million or so broadband users. While broadband analysts Point-Topic predict that number to grow significantly over the next few years, it’s reasonable to expect file downloading to grow at a similar (or, More likely, faster) rate.

Then there’s the problem of identifying the traffic. While it’s possible to roughly identify the application sending each packet, it’s impossible to say exactly what it’s being used for. There’s no point in just labeling BitTorrent connections as illegal downloads, especially as many open source projects use it as a tool for reducing the load on their servers and for giving users faster downloads. There’s also the issue of identifying BitTorrent connections, as many clients connect on random ports and encrypt their connections. To a packet monitor a BitTorrent client delivering a copy of Open Office looks much the same as the latest episode of Doctor Who.

Traffic volume isn’t a signifier, either. That big download could be an MSDN file transfer of an ISO of Visual Studio or Windows Server 2008. It could even be someone using a cloud storage service like Mozy to upload several gigabytes of photographs to a secure online backup service. Or perhaps it’s someone using 4OD or the BBC iPlayer to download a TV programme they missed. With consumer DSL finding a business role as a tool for connecting branch offices it could even be an estate agency updating its database (with the myriad digital photos a house sale needs these days), or an insurance broker delivering a batch of scanned and signed forms to head office.

The bottom line is quite simple: it’s virtually impossible for ISPs to economically identify and filter user actions that infringe on copyright files. The cost of implementing filters is prohibitive (look at how long Google took to even start filtering YouTube), and the time needed to identify exactly which users do what over the network will detract from actually managing and running a commercial network.

There’s also another part to this story.

Running a consumer ISP is hard enough without having to cope with the additional demands of regulators. Customers are on the phone 24×7 demanding service levels that any business IT department would find impossible to implement. An increasing range of IP connected applications and services are stretching thin budgets to the very limit, as network engineers try to emulate a Star Trek chief engineer putting the Enterprise back together with nothing but string and sealing wax.

It’s an issue I’ve had personal experience with, as I used to run the technical side of UK Online - and that was back when dial-up connections were the norm. If it was virtually impossible then to manage usage, what’s it like for today’s ISPs that see an explosion of protocols and packets across their fragile networks.

Illegal filesharing is the least of an ISPs problems. Legal file sharing is a much bigger problem, as the protocols used by services like the iPlayer are inefficient compared to BitTorrent, and the expected traffic volumes are more than likely to overstress existing interconnect and backhaul bandwidth. It’s probably fair to say that ISPs are now finding themselves squeezed from both ends.

If bandwidth is an issue, then ISPs will find ways of controlling it. Plusnet is traffic shaping to reduce network load at peak times. It also produces graphs to show just what type of traffic is used - and when.

Plusnet also produces indivudual reports for each user to show what traffic they were generating. It’s an approach that helps users schedule their own downloads to appropriate times - reducing overall load, and letting users choose their own quality of service.

If  ISPs aren’t going to feel under the pressure, then may be this is the type of approach that’s needed.

–Simon

Anyone who’s listened to me rant over the last couple of years will have heard me say that I expect Microsoft to takeover Yahoo!. With Google’s dominance over search, and Yahoo!’s success at what Microsoft wants to build into Live, there’s a certain logic to a merger of the two businesses. Yahoo needs the R&D boost that Microsoft can give it, and Microsoft needs the online presence of Yahoo!.

I wasn’t surprised to see that Microsoft has made a formal offer to Yahoo!, offering $44.6 billion for the company.

Microsoft’s been playing nice with Yahoo! for some time. It’s Windows Live Photo Gallery handles uploads to Flick, andWindows Live Messenger can talk to Y! users. The love goes both ways too, as Y! is one of the first applications to really take advantage of the Vista UI enhancements.

Reading Steve Ballmer’s letter to Yahoo! this morning I noticed a couple of quotes.

The first is the one everyone’s expecting. Microsoft needs Yahoo! to compete with Google:

While online advertising growth continues, there are significant benefits of scale in advertising platform economics, in capital costs for search index build-out, and in research and development, making this a time of industry consolidation and convergence. Today, the market is increasingly dominated by one player who is consolidating its dominance through acquisition. Together, Microsoft and Yahoo! can offer a credible alternative for consumers, advertisers, and publishers. Synergies of this combination fall into four areas:

Scale economics: This combination enables synergies related to scale economics of the advertising platform where today there is only one competitor at scale. This includes synergies across both search and non-search related advertising that will strengthen the value proposition to both advertisers and publishers. Additionally, the combination allows us to consolidate capital spending.

Microsoft has been talking about online advertising as a key play for some time. There’s a lot of advertising money out there, and even if you add in the efficiencies of online over other mechanisms, not enough if it is going to the online side. Google may have a lead at the moment, but it’s one that can be eroded. Besides, there’s enough money out there for everyone to do very well thank you.

The second quote is, for me, the more interesting:

 Emerging user experiences: Our combined ability to focus engineering resources that drive innovation in emerging scenarios such as video, mobile services, online commerce, social media, and social platforms is greatly enhanced.

The industry is going through a big change right now, and I thonk it’s one that Microsoft has its finger on. It’s not the move from offline to online - it’s the fact that computing power now has finally reached the point where the boxes on our desks can finally be really useful.

Sure, they can add up and act as glorified type writers. That’s not what we signed up to when we got on the IT train. Management information systems were part of the story, but the real game changer is getting applications to understand context.

Context is a big problem. It’s not just the fact that I’m online now, it’s where I am, what I’m doing, what my diary says, what my friends and colleagues are doing, what my boss wants me to do - it’s everything about me and my world filtered and annotated to help me make decisions. Think of it as SAS-style business analytics about yourself, using self generated social networks from tools like Xobni.

That’s where Yahoo! comes in. It’s been doing a lot of work on tagging and on determining meaning in tags. You’v probably seen pictures of Jerry Yang’s demo at CES, of a map of Las Vegas overlaid with Flickr and Upcoming tags. Yahoo!’s been showing that demo for a while now, and it’s part of the context story: building applications that understand me and my needs.

The endgame is Clippy done right (and on steroids). Instead of an annoying “I can see you’re writing a letter” your copy of Outlook says “I can see you’re going to Las Vegas for a conference. Do wish to contact your colleagues who will be there? Susan has free time in here diary in Wednesday evening. She’s vegan, so here’s a list of suitable restaurants. I’ve also noticed that Jet Blue has a special air fare for the days you want to travel and that will give you enough points for next holiday flight. There’s also a room in your preferred hotel. You might want to pack sun screen - it’s going to be hot.”

Your computer knows a lot about you. Perhaps it’s time for it to  start putting it to use. Microhoo! could just be the way it starts to do it…

Technorati Profile

Next week sees the arrival of the remastered original Star Trek series on HD DVD. We saw a check disk of it a while back, and were astounded at the quality of those 1960s images. You could see the patterns in the mesh of Spock’s space suit in a scene set in a frozen scientific outpost, surrounded by dead showroom dummies.

HD video is starting to go where no man has gone before on its own. We wrote back in May about the plan to place HD cameras a mile or so down below the surface of the Pacific, monitoring black smokers - and delivering the live imagery over the LambdaGrid high-speed academic network (which sadly failed to agree merger terms with Internet2 earlier this week).

Now it’s in space too, as the Japanese KAGUYA (which translates as Selene) lunar orbiter is carrying one of the first space-rated HDTV cameras. The probe is still in shakedown, but has started sending back some spectacular imagery.

JAXA, the Japanese space agency, has turned some of the imagery into two rather wonderful movies - one of Earthrise, and one of Earthset.

We’ve grown up with grainy episodes of Star Trek and even grainier Apollo television pictures, It’s good to finally get a HD look at another world for the first time - whether it’s a mile below the ocean, or a quarter of a million miles away, orbiting a hunk of rock…

HD’s on its way to the Internet, too. Microsoft’s Silverlight supports HD codecs, and Flash will soon join the HD scene (just in time for YouTube to decide one way or the other). It’s also on the way to the familiar DIVX video codec, as the San Diego company just bought a German codec development house that specialises in H. 264. Whether we’re using Silverlight, Flash or DIVX it looks certain that we’ll be looking at some form of HD video.If only we had the bandwidth to deliver it to the home…

My trip to IFA was like one of the jokes that go on for hours.

The good news was I saw some really interesting products, had an argument about wheher Dell counts as innovative (feel free to convince me but I see them as the ultimate, though good value, beige box), found the hotel Internet connection well set up and used it to get checked in ready to come home. I still consider 22 euros for 24 hours somewhere the wrong side of extortionate (17 euros if you don’t want to connect a second laptop or a PDA and you know you won’t need more than 400MB of connection), but what I really liked about the Swisscom setup is that every time I connected in the room it reminded me of my user name and password for using wireless in the lobby and suggested I drag it as a snippet to the desktop for reference.

The bad news was that checked in and stuffed with tidbits of information, like Toshiba promising to add HD DVD drives to low-end notebooks like the Satellite range, I got to the airport to discover the flight home had been cancelled.

The good news was I got through to the travel agent and rebooked the flight while I was still 70 people back in the queue for the ticket desk (because I’d copied the number of the person to call into the appointment in Outlook, so it drifted automatically across to my phone).

The bad news was the new flight was at 6.15am and went to Stansted rather than Heathrow.

The good news was I could go home by train.

I want to say that technology saved the day, because I found the details of the overnight train to Brussels and the onward Eurostar to London on the excellent Seat 61 site, with pictures of the sleeper coaches and departure times from the two main stations in Berlin. The bad news was that it was too late to book a seat online or by phone and despite being told that there were plenty of sleeper berths left, I rushed off to the train station and discovered there weren’t any. Obviously there’s a limit to just in time systems, but if an airline can have a seat map for checking in, why can’t a sleeper train show me whether I’ll be able to get any sleep.

Back to the airport and things turned into a species of farce with hotels that had had no rooms when I left discovering plenty of rooms, a shuttle bus that could only be summoned from an office that was now closed and a taxi driver who logically enough presumed that however mad the English are, the three of us couldn’t possibly want to take a taxi to a hotel right next to the airport and took us to the next nearest hotel of the same name. By the time I got the right side of dinner I’d been to the same airport three times in one evening.

Despite the train disappointment, the real good news was how much I managed to get done from my smartphone while rushing around; arranging review items, courier pickups and deliveries, phone interviews, flights for our upcoming visit to the US and updating friends and family with my swiftly changing travel plans. The other big disappointment was discovering that despite the runway closure that caused the cancellation having hit the BBC News site before lunchtime, BA hadn’t notified me at any point before I got to the airport that there was any kind of problem. Given that a weekend flight to Jersey generated four emails offering travel tips and asking for my opinion, just a little more notice could have let me come home by train after all.
-Mary (no longer in Berlin)

Linux isn’t ready for the mainstream notebook but apparently it’s going to be the salvation of the mobile market. Even if you’re convinced by that, the question is - which Linux?

Intel is backing the Moblin initiative for an interface layer on Ubuntu and Red Flag - to my mind a good way to go because interface is key for mobile devices; compare the front ends on both the HTC Touch and Origami launcher for UMPCs which are perfectly designed for the thumb, and the shock of getting dumped into the standard, stylus or mouse proportioned OS underneath. 

Nokia has Maemo, which hasn’t got much of a foothold but does think about the touch interface. If you tap the same input box with your finger instead of the stylus you get a keyboard sized for your finger the usual stylus-sized keys and menus also scale up for fingers and down for the stylus. That means applications can automatically adjust to the way you’re using  the device without the developer having to code that up every time  - and it’s the applications need an appropriate UI as much as the OS navigation does.

Trolltech has its own platform as used by Motorola in the Ming. We could count Mac OS on iPhone as it’s a BSD at some remove. And now ARM is setting up the Linux Mobile Computing Platform based on Gnome Mobile and an as-yet unannounced distribution. Unless that’s one of the  ones listed above and Gnome Mobile works with the other interface layers, you know what you’ve got? YAMLP.

The ARM initiative is aimed at the traditional mobile phone folks, from Samsung to Marvell and Texas Instruments running an embedded system so you can run the same apps on lots of different phones, which Java phone game developers would welcome with open arms. Intel and Nokia are starting with Linux for what I think of as next-generation Web pads, which is what the iPhone really is but Intel is planning to push down into the phone space as well if it can get the power budget low enough. What makes a smartphone a computer rather than an appliance like a feature phone isn’t the operating system that it uses or the built-in features - it’s whether you can install the applications that you choose. That’s where the arguments about the iPhone really start; you can dig around and interrogate the platform and derive the API, and you can install applications - and Apple can send can update that turns any iPhone hacked to run those applications into a pretty paperweight until you ask for a new SIM and start again.

Windows based UMPCs are too big and too clunky and the batteries don’t last for long enough. But you can run all the applications you use on your standard PC. They may not look as good and we badly need an adaptive interface for Windows that deals with pen and finger as well as keyboard and mouse. And you absolutely need to remember that like any portable a UMPC isn’t connected all the time and probably has low bandwidth when it is. But it’s a PC with all the richness of the PC ecosystem. All these other platforms will have their share of applications - but if there aren’t enough applications to make Linux the dominant desktop platform, how does having this many variants of distribution and interface layer for mobile devices that are already a challenge to develop for make Linux the best idea for ultramobility?

– Mary

It’s Monday, we must be in North Carolina. We are, in fact, visiting SAS and this morning we had a fascinating conversation with Dr Jim Goodnight, the founder and CEO. Two things he said particularly struck. One is that many of the companies doing business intelligence are really just looking up details in a database and making it look pretty while SAS focuses on what he calls the ‘heavy lifting’ - true business analytics that model not just what your company is doing now but what your company is about to do. The other was that SAS could give a company a list of the customers they would lose in the next three months - and a report on whether they’re profitable customers you want to hold on to or time-wasting makeweights you’re happy to do without.

This afternoon we spent an hour on the phone to Virgin Atlantic all told, and for far too much of it I felt distinctly like a makeweight. Planning ahead for our next US trip, we wanted to book flights to CES in January and do it using all the miles we’ve accumulated this year. Not only will we want to see all the new gadgets, but McCarron Las Vegas is putting in the same RFID luggage tracking system used at Hong Kong airport - a much needed upgrade that might stop the luggage carousel grinding to a halt every ten minutes - and it will be interesting to see if that changes any other airport business processes.

Problem was, Virgin hadn’t credited the miles for some of our recent trips, leaving Simon 27 miles short on the total. we rang up; could they transfer miles between us? No. Could they credit the miles we were both missing? Tap, tap, tap; hold please. Mary’s flight was credited instantly and as it was all going to well we clicked the button to book her flight. But Simon’s flight, the one that actually needed the miles? Hold please. Hold some more please. And hold one more time: sorry, it’s not coming up on the system, did he actually take the flight? As it wouldn’t come through and the support team had gone home, why didn’t I phone the US office. Ring, ring; we explain again and emphasise that we’re in the middle of booking the flight online and don’t want to lose the booking. Hold please. First I was on hold so long that I got transferred to another agent; then I was on hold for half an hour, on and off, as they said they would check - hold please, no they couldn’t - hold please, I have a message, no I still can’t - hold please, I have another message - no, you have to send an email to do this.

In the end we gave up and bought enough miles to make up the difference, went back to the booking - and found the Virgin site had closed for the night. A Web site that closes for the night! Because you’ll never want to book your next flight while you’re still abroad, or if you’re a shift worker or just an insomniac. At this point I lost my temper and when I discovered that the main Virgin number was also closed phoned the Upper Class number and asked them to solve the problem.

If I’d had the same runaround from Virgin I’d had on the other three calls I’d have been off to the British Airways site even though they don’t fly to Las Vegas and don’t let you fly in different classes on each leg of the ticket. I might even have remembered that we had planned to try out MAXjet who can be as little as twice the cost of a Miles flight on Virgin for a seat that reclines to 160 degrees , although you have to get out to Stansted. But I got an extremely helpful agent who didn’t quibble or ask if I was accredited to phone the premium number or say I should try to book online, whizzed through the booking, sorted us out seats together - and charged us less for Simon’s ticket counting the fuel surcharge and taxes converted out of dollars than I’d paid on the Web site.

I don’t know whether to wish that Virgin was using SAS instead of the class of Flying Club card you have to decide who gets helpful service on the assumption that I fly enough to be a valued customer - or be glad that they weren’t because that way I couldn’t get out of being classified as unwanted. I do wish they’d either get with the 24-hour society and flexible working or just credit us the miles we’ve earned to start with. My time is valuable - but I’ve also just cost them nearly an hour of not helping any other customers either. An intelligent business wouldn’t be wasting that money.

The latest DARPA Grand Challenge is nearly upon us, and Stanford University’s entrant made a rare public appearance at IDF. Stanford won the previous event, and are among the favourites for this year’s race.

Racing across an empty desert is easier than driving in traffic, and though Stanley managed to overtake one of its opponents, it wasn’t designed for the open road. Junior is a different kind of robot, built into a Volkswagen estate car, and covered with sensors. Its task is to navigate down the open highway, mixing with real traffic. It would be an understatement to say that this is a hard problem…

At a first look, Junior doesn’t look that different to one of Google’s street mappers. That’s not surprising - the equipment racks come from the same source. The sensors also serve much the same purpose. A differential GPS pinpoints the car, while LIDAR arrays find out just how far it is from objects around it. Meanwhile, a high-speed spinning camera looks at the moving objects around it. A display in the car showed just how effective that view was, showing people milling around the car and the show booths.

Two racks of servers sit in the boot, ready to navigate the car through the simulated streets of the Grand Challenge course. There’s a lot of power in there - Intel’s support staff told me that they’d managed to halve the amount of computing hardware in Junior, but still managed to give it 3 times the power of Stanley. That’s power it needs - there are many more sensors, and a much more complex environment to work in.

The Grand Challenge is intended to encourage the development of the tools and technologies that would power military robot logistics vehicles, carrying munitions across dangerous terrain, in completely autonomous convoys. There’s plenty of civilian options, too. It’s easy to imagine using Junior’s sensor arrays to give drivers more information, and to warn of (and even protect from) impending accidents.

Tomorrow’s cars will mix automatic and manual driving. It’s easy to imagine getting on to the M40, putting one of Junior’s descendants into automatic, pulling out the laptop, and getting on with some work. There’d be no need to worry about the roadworks, or the traffic conditions - your car would do all that for you, leaving you more time to clinch one more sale…

Here are some pictures of Junior:

Not your average Volkswagen. Junior pays a visit to IDF.

How Junior sees the world. You can see the vehicle at the centre of the image, with people and show booths around it. I’m the white blob in the rear quarter of the car, crouching down and taking a picture through the door…

Half the rack space of Stanley, three times the power. All you need to go play in traffic…