Originally published on semanticweb.com

A few months back I posed a question to the folks at DERI (Digital Enterprise Research Institute) from the University of Ireland when they came to visit Radar Networks. This is a question that I have struggled with for a long time; seeking for answers anywhere I could. When I asked them, I heard the same question in response that I always hear.

Why? Why would you ever want to delete something out of the ontology?

Ontologies were not originally created to have things removed from them. They were created to classify and organize information in a relational format. Just because a species goes extinct it doesn’t mean it should removed from the domain of animals does it? Just like a car that isn’t manufactured anymore or a disease that was officially wiped out. These and many more are probably the reasons why Semantic Web gurus and ontologists alike don’t like the idea of deleting entities.

I am helping to create a social site where users generate content; objects, notes, data, and connections to others and other things that are their own. If they want to delete something that they have created, so be it. Sounds easy right? Well, yes and no. This problem is dealt with throughout computer systems. It is essentially all about managing pointers in memory. You can’t delete something that other things are pointing to. Who knows what will happen or how our application will respond when certain pieces of information are missing? Some things we account for on purpose because they are optional — but some things we just can’t. Every application has its unique constructs, whether it is built on a relational database or a triple store.

So what I have to do is define ontological restrictions, stating what links can and cannot be broken. On top of that, we must worry about permissions. Am I allowed to break an object link from someone else’s object to my own? Also, what if the object being deleted has dependent objects that cannot exist alone, or more importantly, don’t make sense on its own? A great example of this is a user object and a profile object. There should either be zero or two, never just one.

My friend and coworker Jesse Byler had dealt with a similar problem in the application tier a few months back regarding dependent objects. He had written a recursive algorithm that would spider the graph until it hit the lowest leaf node matching certain criteria and then begin to operate. I took this same principle and pushed it down into our platform and began to mark restrictions in our ontology.

This is where it became tricky. Some relationships are bi-directional and some are not. In the user and profile example above the relationship is bi-directional; however, many of our relationships are not. An example to this would be a group and an invitation to that group. Sure the group can exist without invitations, but the invitation cannot exist without the group.

All in all, the design pattern isn’t overly complex, nor are the ontological restrictions. But as a whole, it makes for an interesting problem with a lot of nuance. Careful consideration must go into this process because mistakes could be catastrophic. Data integrity is paramount and dangling references could leave the application in a terrible state due to one bad linkage. Although simple in practice, the execution is anything but.

And for those of you who are still asking yourselves why?

Here’s the answer. Scalability. As if working with a triple store wasn’t hard enough, keeping useless data around that will never be used again will definitely make matters worse. We are attempting to build a practical everyday application — not classify organisms. Surely there is a place somewhere in the mind of the ontologist where he can think practically about using ontologies for data storage. Isn’t there?

As more applications are built using OWL and RDF, this problem will become more and more real — and there’s nothing the ontologist can do about it but adapt, or die a little inside. Either way, at the end of the day, I am still an engineer trying to make do with what I have.

If we must delete, then so be it.

Being a long time geek and open source advocate I’ve finally decided to start an open source project for home automation. I’ve always dreamt of being able to wake up to my curtains opening and my heat turning on rather than the usual “beep! beep! beep!” that beckons me to work each day. How about checking on the house when I’m away or open the front door while I’m in the back of the house. The possibilities are endless and my ideas are just beginning to flow.

The project is called Open Automator and it can be found on Source Forge’s site. Its still a very basic framework but its coming along well. It works using the Insteon protocol that communicates over the power lines in your house. Its very similar to the X10 of yesteryear but with updated speed, reliability, and redundancy. It costs a little bit more but it will be well worth it. These devices can be found all over the internet from sites like smarthome.com.

This first rendition will be geared toward the iPhone and Simple Home Net’s EZServe which communicates via an XML socket connection. That being said, I will be making this application very flexible allowing for many different types of implementations, not just Insteon either. The application is Java based with Maven being used for building and dependency management. This will be coupled with Spring and Hibernate to manage database and model view support. The final product will have Jetty and some sort of SQL engine rolled in to make the application extremely portable and platform agnostic.

I will be writing a lot more about this project in the months to come as well as recording some online videos about how it works and how to set it up. I would like to make this product as simple as possible so its usable by the masses. Just because its open source doesnt mean its going to be complicated. If you are interesting in being a developer please let me know.

Today I came across an interesting concurrency problem while deleting objects from the Social Graph (Semantic Web remember?). I have been tasked with mass deletes throughout our system, including exporting the objects in case they ever need to be reassembled again. Since our graph is so large, and we could potentially be deleting 10’s of thousands of triples at a time, the serialized XML would be about 10 times that many lines per triple represented in a file. In order to write the output to a file as fast as can be there was no need to store the serialized XML in memory. The best thing to do was to pipe the output stream to our binary store.

Now in order to do this, I need two threads, one to write and one to read. If you were to do this with one thread you would most likely run into a nasty deadlock situation. Anyway, here’s what I came up with:

public InputStream openStream() throws IOException {
    final PipedOutputStream out = new PipedOutputStream();

    Runnable exporter = new Runnable() {
        public void run() {
            tupleTransformer.asXML( tuples, out );
            IOUtils.closeQuietly( out );
        }
    };

    executor.submit( exporter );

    return new PipedInputStream( out );
}

Can anyone see the problem? Well unfortunately I couldnt either for over an hour. My unit tests would pass sometimes and fail others which led me to believe I was dealing with a timing issue. Turns out, sometimes the PipedOutputStream was completed before the PipedInputStream was even instantiated, completely missing the stream of the out.close(). The trick was to instantiate the two streams, in and out, at the same time then start the output with another thread. Problem solved.

Today I came across an issue at work regarding the made up MIME type of ‘image/pjpeg’ while uploading images from IE 7. Java’s ImageIO was choking while trying to upload this type of file because it is not in the list of supported MIME types. The supported MIME type list can be viewed by invoking ImageIO.getReaderMIMETypes(). After reading many blogs of others pounding their head, making fun of Microsoft, and using other solutions like ImageMagick, I found a solution. Add this to your list of supported types and move on. This format is made up and will not cause any problems. Pass the input stream onto your File IO and be done with it. Microsoft owes me half an hour of my life back.

Up until recently I never had to write any code that dealt with reflection. Before now I worked for CNET which is basically a series of differnent CMS’ with one request, one response. My first week at the new job required me to build a JMS system that would accept method invocations across the wire seamlessly for the producer. Java.lang.reflect was the answer.

Once you can wrap your head around the idea its pretty simple to write. Essentially what we are attempting to do is shield the caller from having to know anything about the other method that is going to invoked behind the scenes.

public Class MyConcreteProxyClass implements InvocationHandler {

   Object obj;
   public MyDynamicProxyClass(Object obj)
     { this.obj = obj; }

  public Object invoke(Object proxy, Method m, Object[] args)
         throws Throwable {
     try {
        // do stuff
     } catch (Exception e) {
        throw e;
     }
        // return something
     }
}

Now we have to create the proxy interface which the caller will implement. Any of the methods of this class that he implements and fire will invoke method on the MyConcreteProxyClass object.

public interface ProxyInterface{
    public Object myMethod();
}

Now its time to create wire everything up. There are several ways to do this which help hide this proxying like a static factory method but this simply demonstrates how this works.

ProxyInterface proxy = (ProxyInterface)
Proxy.newProxyInstance(obj.getClass().getClassLoader(),
         Class[] { ProxyInterface.class },
          new MyConcreteProxyClass(obj));


This is now one of my favorite books on Java which I am probably going to read again just to be sure I have soaked up as much information as I can. This is a practical book written by a practical person who understands his audience, engineers. He is straight to the point with his code examples and doesnt bore you to death explaining every little detail or referencing lines of code and functions 8 pages back. I wish more technical writers would be this concise and straight to the point. There is something to be said for any technical writer who can get their message across with fewer words. No where else can I think of a better example of the old adage “less is more”. I would highly recommend this book to any engineer, especially associates.

As I have said before I love Spring and Hibernate; however, there is no Spring support for Hibernate’s pagination API. Why am I am not sure. Maybe they are supporting it now but heres what I did. First I made and abstract object I called ‘Page’. I also have a Solr subclass of this object but I will cover that later.

public abstract class Page {

        return this.results = query.list();
    }
}

Since Spring abstracts your Hibernate interaction with HibernateDaoSupport you dont have to write straight criteria calls to Hibernate if you dont want to. Spring does create simple implementation to allow to write straight HQL and many other functions but not pagination. Rather than tackle this problem at this angle I decided to make the abstract class ‘Page’ that you see above for all my data stores. Anyway, here’s how easy it is to then hit the database with this object properly constructed and have it returned ready for use.

public Page getPageOfThings(int page, int resultsPerPage, String orderBy) {

    PageHibernate newPage = new PageHibernate(
            ”FROM Thing”, page, resultsPerPage);

    getHibernateTemplate().executeFind(newPage);

    List count = getHibernateTemplate().find(”SELECT count(*) FROM Thing”);

    newPage.setTotalCount(Integer.parseInt(count.get(0).toString()));

    return newPage;

}

Spring and Hibernate make Java web application development so much easier I barely remember what I did without them. Despite the steep learning curve, Hibernate especially so, they work so well together thanks to Spring. Spring is probably the most “open” open source framework out there for Java. It can and will play nice with tons of other frameworks like Tiles, SiteMesh, Velocity, JSF, as well as backend technologies like iBatis, Hibernate, ACEGI, and countless others.

I have begun to like them so much at the office that I am using it a home in all of my side projects. This blog will hopefully shed some light on some of these technologies to hopefully give back. Blogs like this are especially important because although these frameworks aren’t relatively new, there can be some holes in the documentation. Two books that helped me through my trouble were Matt Raible’s book “Spring Live!” and “Hibernate in Action”