Newsworthy Math
A Discussion about Mathematics in Society
Physical Data
RDF stores in hardware.
Whatever else it turns out to be, the semantic web has already given us RDF. It’s here, it’s now, it rocks.
RDF says that every data structure can be modeled using three part statements consisting of Subject, Predicate and Object. Each of these being a simple piece of data that can be represented as a string.
This is a surprisingly substantive claim. It means two part statements (Subject/Property, or Name/Value) are not enough, and four part statements unnecessary. It means that in structured data RDF triples play the role that triangles play in graphics. If you can draw triangles you can draw everything. If you optimize your triangle drawing you optimize all your drawing. In networking TCP/IP packets play a similar role. Routers, switches and network cards don’t care what new applications are built on top of packets. Make passing them efficient, and everybody benefits, the cost of switching packets tends to zero.
The completeness property of triangles in graphics makes graphic cards rewarding – custom silicon that solves a specific problem insanely efficiently. RDF has similar potential. RDF can scale in hardware. The memory storing the statement can be smart enough to parse and query it. The structure is so simple, basic filtering so straightforward, that it can be done on a per triple basis in hardware, implemented directly in transistors alongside the non-volatile memory, built scaleable to plug into an RDF bus. As you add data you add processing power. Since one of the sweetest properties of RDF is the fact that it can be munged together indiscriminately – you can always add more triples without compromising what’s already there – this means we can have physical data. Physical data scales really well.
I apologize for using this approach to make contact, but I haven’t been able to find your email elsewhere on the blog.
I read you are working on converting the collective intelligence algorithms to F# on a couple other blogs out in the cloud. I had actually set out to do that exact thing a couple weeks ago, and have been working to build a solid foundation in F# and functional programming before I get started (I’m a professional C# developer, but haven’t had any ‘true’ functional experience until now). Since you are already knee deep in this project, I was wondering how it was going, if you would like some help, and if what you have done so far is currently available?