ramblings on PHP, SQL, the web, politics, ultimate frisbee and what else is on in my life
[1]  «  10  11  12  13  14  15  16  »  [71]

DeltaQuery with Solr DIH

Recently it seems like pretty much all projects here at Liip are using Solr in some form or another. Never thought that all my PHP projects would end up relying on Java. This is just a short blog post to document an approach I came up with to make delta updates more efficient and at the same time easier to maintain than the "standard" approach explain in the docs. The Solr DIH handles importing documents from external data sources. In my case I have about 5GB of data in an RDBMS that gets updates every night. The standard approach in Solr is to define one query for the initial import and a second query to fetch the IDs of documents that have changed and a third query to fetch the data that changed. Especially if you expect a large number of changes this isn't very efficient (*). Furthermore if both in the initial import and the delta case you have the same SELECT list, its tedious to maintain 3 queries where two are almost identical and one still very similar.
read on (comments 2)

FrOSCamp: Symfony2 CMF and CouchDB ODM

So thanks to the sponsorship of Liip we managed to get quite a few people from outside of Switzerland for some pretty exciting meetings to Zurich for FrOSCamp. Plenty of Liipers were also present. On Friday we sat together to learn and talk about Symfony2 CMF. On Saturday we split up in two teams, one working on the JCR client library Jackalope and the other starting with adding CouchDB support to Doctrine2.
read on (comments 3)

Oracle needs to get hurt .. badly!

There are plenty of bad things in the world. But I guess there is nothing that has such a drastic and dangerous impact on my life that software patents. If patents are good thing in other industries can be discussed, but in the software industry its a clear cut thing: they hurt innovation, they hurt small businesses and they scare the shit out of me. And if you are a software developer, they should scare the shit out of you too! Now Oracle decided as the first big company that is not just a patent troll to actually sue a company over software patents. heck maybe others have sued before, but lets stop this behavior right here right now. We must send a clear message. We must send a clear message to Oracle that they better stop. And we must send a clear message to any other company holding software patents that they better not think of suing. I have not thought this out all the way yet, but right now I am thinking the best form of protest would be if all of us open source developers strip out support for any Oracle product in our next release: Throw out support for BerkleyDB, MySQL, Oracle whatever. The business math must be clear: Suing over software patents will cost more then it can ever get in returns. Who is with me in organizing such a protest? Or does someone have a better idea of how to approach this?
read on (comments 16)

Red tape and the art of ripping through it

It seems like the Symfony2 CS is forbidding the use of private methods. I think other frameworks have similar rules. Now I have argued along those lines myself in the past. But especially talking to the Doctrine2 dev's got me to reconsider .. a bit. This all kinda of reminds me about the discussion over making it a fatal error to change method signatures during inheritance. I love PHP for allowing me to do this, yet giving me an E_STRICT, aka red tape, to make sure I am aware of the kittens that are being strung up this very moment. When I have coded myself into a corner, I need to get out of it quick, I do not need
read on (comments 3)

Transforming end user queries to Solr

A bit less than a year ago I last did a presentation about a telephone book application where we used SQL to do some fairly advanced filtering over about 30 tables of data. The app generated SQL statements that filled pages, the more terms the more pages, but on a 10k dataset it still came back within a few milliseconds, thanks to a ton of indexing and denormalization tricks (SQL Server is a lot more powerful here than MySQL) I had applied. Now in a more recent project I am dealing with 10M+ dataset running on MySQL and so decided to learn about Solr. Wow, that thing is amazing and way more flexible in terms of query language than I expected. As a result I do not see it any more for just projects that are too big for an RDBMS, but more as the way to do search in general. I have mentioned resolutionfinder.org a few times (used to be called UN-informed.org). Solr is a key piece there and more importantly I am looking to expand the use of Solr query language quite a bit. Actually for those who know, you can already do a lot more powerful queries, something Liip will be investing some more time to make more accessible to end users with some UI tweaks planned in July. But in this blog post I want to talk about a prototype class I threw together (Look ma', I'm using git!) by working ezcSearch to help me in parsing and transforming end user queries into complex Solr queries.
read on (comments 2)
[1]  «  10  11  12  13  14  15  16  »  [71]