What's My Process for Creating APIs

@fogus asked in interesting question this morning:

How do lang/lib creators know what's best for the average programmer when what they try to solve is nebulous or underspecified?

I responded:

@fogus I throw stuff up against the wall and promote what sticks. There’s plenty of fail experiments I’m Lift.

But that's not the whole story... so, I'll outline some of the processes that I use to create APIs and code.

I'm My Own User

Much of the library/API code I write is for me. Or more specifically, for projects I'm getting paid to work on. Lift to a great degree grew out of work I did on Innovation Games.

I brought Buy a Feature to life and along with it, brought Lift to life. Buy a Feature is a multi-user online game and it's entirely HTML browser based and had to work with IE6.

Most of Lift's Comet features are there to support real-time pushing of data between game players... in IE6.

So, many of Lift's early features came from doing Buy a Feature.

I Love Going Against the Grain

I'm a trouble-maker. I like going against the grain. So, when the whole world is MVC, I look for something different. Specifically, I look for ways to make the APIs reflect the way developers work. Thus, Lift is "view first" because developers do mock-ups first and fill in the functionality. Rarely have I seen a developer "write a controller on a white board." Nope, they lay out the screen on a whiteboard and go backwards from there. Why not have code the reflects that.

Stealing and Improving

I love stealing other peoples' excellent ideas... especially when those ideas are outside the realm I'm working in.

One of Lift's best features, CSS Selector Transforms were stolen from Enlive because @kirindave pointed me to them one day, and I realized "that's the way to do HTML transformation!"

So, Lift's GUID associated with server-side function is just a web representation of references. Why carry state around in the web page, when it can be carried as a reference to a function that closes over the state server-side. It's just a way to treat Scala functions the same way across address spaces. It's also a Scala/Lift rip-off of Seaside's Continuations.

Security

I am a security nut. Part of what drives my API designs is ways to make it easier for the developer to write secure apps by default. So, when I'm writing APIs, I try to figure ways to keep data reified until the last possible moment rather than converting things to Strings or other un-typed data early. But this has to be natural for the developer.

Trying Stuff Out

Sometime, I try stuff out, like Lift's Wiring because it seemed like a good idea at the time. Sometimes people use it (like Wizard), and sometimes, like Wiring, they don't.

Watching the Mailing List

I also watch the Lift mailing list and see what kinds of things people are trying to do and then write APIs for common threads of what other folks try to do.

Documentation

I wrote Lift's RestHelp by writing the API, documenting it, seeing where the documentation looked ugly, and revising the API to make the documentation cleaner. It was an interesting 3 days of solid "how can I make this cleaner and easier to understand?"

I think this is the most virtuous way to write APIs and I hope to do more of it in the future.