David Pollak's blog... and such.Lots of DPP's thoughts here...
October 31, 2014
It's a Virtual Piece of Cake
I've gotten a bunch of questions about how Visi, the simple web front end to Spark works. This blog post is an overview.
Hosted Spark with a Simple Front End
Visi is a hosted Spark cluster with a simple web-based front end that allows Excel-savvy folks to enter formulas that get turned into Spark jobs.
The Spark cluster and front end are built on demand, hosted in Docker containers, and communicate over the network using Weave. The web UI is presented to the user via a dynamically updated HAProxy routing table.
Nuts and Bolts
October 29, 2014
Back to the Future...
This post was from November 2006. I had just started playing with Scala and was trying to figure out an ORM... the one that ultimately became Lift's Mapper.
Keeping the meaning with the bytes
One of my criteria for a good web framework is having security and access control built it. As I was driving friends and relatives to and from Thanksgiving dinner, I was thinking to myself, "It's nice to have goals, but how do you implement then?"
September 24, 2014
Blast from the past...
I wrote this in May, 2008... and I've gotta say, I was pretty spot-on including Java 8 adopting some of Scala's better features:
It's starting to happen... the FUD around Scala. The dangers of Scala. The "operational risks" of using Scala.
It started back in November when I started doing lift and Scala projects for hire. The questions were reasonable and rational:
August 30, 2014
It's the chat demo
Check out the demo!!
As I've been working through the Dragonmark stuff, I decided to use the same Chat app as a demo. Why? 'cause the same concepts are present in Dragonmark... the abstraction of the cross-address-space plumbing.
core.asyncacross address spaces
August 16, 2014
Distributed Communicating Sequential Processes (CSP)
However, very few programs run in a single address space. Web applications run in a combination of the browser and one or more servers. Very often, applications will span a cluster of servers.
Dragonmark Circulate provides a mechanism for distributing
core.asyncchannels across address spaces while providing the same semantics to all the address spaces.
I've written some macros to make writing
core.asynccode easier and more linear.
July 22, 2014
I've started working on a series of open source library code for Clojure called Dragonmark that roughly falls into three categories: utilities, a distributed CSP library, and a sample web app that demonstrates distributed CSP.
Mostly, I think that the semantics for interprocess communication should be the same as the semantics for local communication. Clojure's core.async library provides a really nice set of APIs to communicate asynchronously, have backpressure, and in general "do the right thing."
July 14, 2014
Not my cup of tea
I went to Yes & over the weekend and it was disappointing. I was drawn in by:
The theory was, if you get a group of smart, engaged individuals together in the right place that great things would happen.
Sadly, the conference was not that.
I was expecting more of a Strange Loop experience. Something is a far away place where each interaction leaves me thinking, pondering, and expanded.
Instead, Yes & is a big, fun partay.
If you'd like to hang out in the pool, get drunk, share new strains of medical marijuana, and go to a fun prom with all the hipsters from the Mission, Brooklyn, LA, and Portlandia, Yes & is the place to be.
May 14, 2014
When we use Java
There is a lot of confusion about my We all Infringe post. So, I'm going to walk everybody (especially the lawyers) through the mechanics.
Until last week, all developers and most lawyers operated under a simple rule. APIs are not subject to copyright, but code implementating APIs is subject to copyright. It was a simple dichotomy and allowed developers to operate under a brightline set of rules.
On May 9, 2014, the Circuit Court of Appeals changed the law to include copyright coverage to include APIs. Specifically (page 5):
... we conclude that the declaring code and the structure, sequence, and organization of the API packages are entitled to copyright protection...
May 12, 2014
Okay, everybody who touches Java bytecode
The Oracle v. Google holds that copying the Structure, Sequence, and Organization of the Java APIs is a copyright violation. And a copyright violation is not just the act of copying, but also applies to all the intermediate parties that have a copy of the work.
That's anybody who writes/compiles any JVM language and anyone who has a JAR file on any device they posses... including a Java ME applet on your old Motorola flip phone. In fact, the JVM in all its incarnations is so pervasive, it's likely that every adult in every industrialized nation has some JVM running someplace.
May 11, 2014
I have been developing software professionally since 1978. I went to law school (BU Law '91). I think that computer programming technology and the law are really, really similar.
At the end of the day, both law and computing is about wrapping abstractions around very complex interactions such that the rules are comprehensible and the outcomes are predictable.
May 11, 2014
He's not So Fine
Sometimes, copyright law can have really bad side effect. It's supposed to help content creators make money from their content and that's awesome. But in the George Harrison case, he lost a copyright suit because he "subconsciously copied" another song. If this is the standard that the court is going to apply to all API design, the computer industry is screwed... luckily starting with Oracle.
We build on the shoulders of giants
May 10, 2014
Are APIs Copyrightable?
Oracle v. Google resulted in a really bad opinion for everybody in software, including Oracle. The Federal Court of Appeals applying 9th Circuit law held that the structure, sequence, and organization of APIs are copyrightable. This is a radical change in the law as to date, everybody and everybody's lawyers have taken the position that APIs are not copyrightable, but the implementation of the APIs is.
First the mitigation.
Are long as the APIs are reverse engineered, it's okay to clone them
April 26, 2014
Yeah, it's not Open
I have a greenfield project where the choice of database is open. I have been very interested in Datomic for a couple of years now and figured that I'd try Datomic out for the project. Then I tried to actually download and install it and decided against it... mostly because the value of Datomic does not clearly outweigh the cost of using proprietary software. I am writing this post to communicate my perception.
People who write software should be reasonable compensated for their work. I write software... some open and some proprietary... and I like to be compensated for my work.
April 6, 2014
But I am really dismayed by the way the tech industry is handling both the inclusion of under-represented folks as well as the way the industry gets into mob mode when someone is "outed" for funding a non-mainstream view.
What the Underlying Issue is About
March 12, 2014
The Lure of Private Schools
I read this post when I woke up this morning. I am a huge fan, supporter, and believer in public schools and specifically public schools in San Francisco. I am just totally dismayed at people in my cohort putting their kids in private school.
I get the mindset of a people who have money and normally use money to buy solutions to problems. There are some problems for which money is a very reasonable solution approach. Want a quiet car that doesn't have technical problems and when it does, they get solved with little time/effort on your part, you can buy that. Want flawless, high quality construction on your house, you can buy that.