Like I mentioned in my last blog post, we have a company internal framework that is quite advanced thanks to the various symfony components we have integrated. Its other main advantage is that it can do a lot with very little code. The net benefit of this is that it's extremely easy to learn and debug. Especially the last point can be somewhat painful with symfony since inheritance trees tend to be quite large and things get delegated to other objects etc. However once you start getting a hang of where things are, symfony is very powerful indeed. The other day we had a little hackday at Liip where we wanted to write a little google maps app. The idea was to have different kinds of markers shown on the map which when clicking on them would both load some data into the classic GMap marker bubbles as well as some additional data into a div below. Furthermore, the markers should be filterable. We choose symfony 1.4 as the basis, mostly because we wanted to learn a bit more about symfony, but also we felt that some of the module generation tools could help us get off the ground faster.
We are not quite done yet as things ended up taking a bit longer than we expected, but I think we lost more time with the GMap javascript simply not being at the same level as other JS frameworks like JQuery or YUI. Seems like Google has some catching up to do. Anyway, in the end a co-worker gave us some key tips in order to make the GMap-YUI-combination work out, but thats not really my area of expertise anyway. In the end we decided on the following communication flow: The backend would (re-)generate a javascript file whenever a new marker gets added or one gets deleted or if the name, latitude or longitude of a point changes. The js file would also store the type of the marker. This way we could quickly show all markers on the map with an icon to denote the type. Whenever the map is repositioned, we would send the boundaries to the server, who can then run a simple query to fetch the data to be shown in all the bubble currently visible on the map in one request. Then when someone actually clicks on the bubble we would load the additional details to be shown below in another request. The filtering would basically just return a full list of ID's of all markers that should still be visible. We figured that this way we could really limited perceived latency to an absolute minimum: You immediately see the markers are you move the map around. Once you let go we load all the information in the visible bubbles so you immediately get something when you click on one and finally if you are really interested to see all the data on the marker, we have some time to load that data as your eyes travel below the map. Most of this has been implemented by my co-workers Memi and David already.
I was mostly working on the backend and the first thing I did was setup the DB schema, for which I choose the Doctrine yaml format. After a bit of experimenting I settled on using inheritance. This way I would define a base "entity" model, form which I could extend all the various custom data types. This way I could also keep the entire logic for regenerating the javascript file with the map markers in this "entity" model class. However Doctrine 1.x inheritance is a bit annoying. You have the choice of either simply lumping together all fields including all specializations into one table (column aggregation) or to keep the data totally separate in one table for the parent and another one each for all the specializations (concrete). Doctrine 2.x will add another inheritance solution, where you actually store the common data in the parent table and only create tables with the additional data for each specialization.
Since Doctrine 2.x isn't ready for prime time yet I decided to use concrete inheritance with some magic. The key thing was to ensure that if a markers gets added, updated or deleted it would regenerate the cached markers javascript file. With a simple preSave()/preDelete() hook in the entity model it was easy to automatically regenerate said file. Actually in the case of an update I only need to regenerate if the name, latitude or longitude changed. So I added a preSave() hook that would check what properties were modified in the case of an update. Generating the file will probably never be such a big deal, but regenerating the file would also cause all users to have to reload the javascript file which just adds traffic and latency. While we do expect that this file will need to be regenerated daily as people will add themselves or change their location (especially if we ever auto update their location based on Google Latitude), we do want to minimize the risk of having to reload during a session.
The next bit of magic I thought I needed was a common unique ID for all (we thought we needed integer ID's for GMap, but actually GMap needs 0-indexed sequential ID's so we had to create a mapping array instead of just identifying map markers by their backend ID). Since all data is stored in totally separate tables, which use separate autoincrement generators (a good example of where sequences are more flexible). So each type would start of with ID 1 etc. The solution for this also turned out to make generating the marker file easier. I decided to denormalize things a bit, using Doctrine preSave()/preDelete() hooks, I would create/delete an entity instance before doing the change on the actual model instance. In the create scenario I would then read out the generated entity ID and also store it in the type specific model instance (I guess I could have used it as the ID and drop the entire autoincrement column on the type specific models but oh well). Finally I quickly generated admin interfaces for the different types I had defined using the symfony CLI. Very cool.
On the backend logic for the map module I first added a little debug solution for XmlHttpRequest (not sure why symfony still does not have something like this natively, since I hacked in the something similar 3 years ago). Required some additional mucking around since GDownload() does not support custom headers required to have symfony's "isXmlHttpRequest()" work. The key idea is that inside methods that expect an AJAX request it would check if in fact the request was an AJAX request (unless when in debug mode) or redirect to the homepage. In the case of the debug mode it would also output the data nicely formatted instead of plain JSON.
Other than that I needed 4 public methods in the backend: One to load the interface, one to load the bubble details, one to load the additional details and of course one to filer the markers. The first one to load the interface I just created a new form class to be able to generate the filter form. Since the form has some type specific fields it seems to make a lot of sense to define the form inside the backend code and not some template. Since all the methods for fetching data are type agnostic, the only thing inside the frontend code that is really type specific is the different marker icons per type.
Getting the bubble data I decided on issuing one query on the entity table to get all the relevant ID's that are currently visible on the map. I would then issue another query for each type in that list of ID's and use a partial to render the bubble view. Of course I could have also joined the entity table with all type specific tables to get it all in one query. But it would have meant defining those relations inside the schema file, which could potentially really screw things up in the generated admin tools. Also doing a LEFT JOIN per type isn't really the kind of thing that will make MySQL happy especially if we add more types down the road. Similar situation inside the method to get additional details, where I would again first query the entity table to find out which specialization table I need to query. I guess here we could in theory also pass in the type as part of the public API to get around that first query.
The last method of interest is the filter method. Here I decided to have again one query for each type to keep things simple instead of trying to stuff everything inside a single JOIN. Doctrine does not support UNION, but I guess I could also use subqueries, but MySQL isn't the best performing when it comes to subqueries either. Anyways for now I was not so much concerned about optimizing this to the last bit. Doctrine's fluent interface to build up SELECT queries is a bit limited when it comes to grouping expressions, so I am using a different default query class which gives me the ability to atleast group all WHERE conditions once per query using the custom "whereParenWrap()" method. Other than that it was all quite straight forward.
The last thing I wanted to add was using the "swWidgetFormGMapAddress" widget in the administration of the different type data, so that users didn't have to type in the latitude or longitude and could also verify their address. Again since I am using inheritance all I needed was to adjust the entity form class. Basically I remove the "address", "latitude" and "longitude" widgets while adding a new "map" widget. Then when it comes time to store the data I move the data out of the "map" widget back into the model instance being administered. I also hide to "entity_id", "type", "created_at" and "updated_at" widgets, because thats all really just internal stuff.
That is it for now. There are still a few things left to do. Mainly the filter method needs to be AJAX-ified. I also want to center the map according to the viewers location by default (using the browsers geo location API if available or an IP-location DB as a fallback) as well as allowing to position the map by entering an address. We are also missing the final list of types and properties and some design polish. More importantly we do not yet have any sort of security concept to decide who can edit what. At any rate I am really liking how things have worked out. Most of the ugly hacks I added could eventually be removed after digging a bit more in the excellent symfony documentation (only gripe is that its not searchable in the site and when doing google searches you get flooded by 1.0/1.1/1.2 docs while we were using 1.4). I am also quite happy with how using Doctrine inheritance made things so much easier. Hopefully we can put the finishing touches on this app soon so that we can go online early next year.
