Back when I was developing MDB2, I loved my unit tests to help me ensure that changes actually worked across different RDBMS (versions). It was also a great way to identify issues by having users run the test suite. Actually I guess I wasn't really "unit testing" since I was actually always running against a real database. Fast forward until today. In my current project we are aiming for 100% code coverage. Well we are not afraid of using @codeCoverageIgnore for super trivial stuff and yes I know that there is more to good tests than just coverage. At the beginning we actually unit tested everything. So this meant writing lots of mock object boiler plate code for Controllers for example. Pretty soon it became clear to me that we weren't unit testing API's, due to all this mocking we were actually testing the implementation.

I was mocking each and every method call to any dependency with the parameters being passed and the return values. If I changed the code without changing the API at all, I would still need to update the test. That felt seriously wrong. Furthermore while unit tests are great to spot the exact method that might be broken, the reality is that during a normal PHP web request there aren't that many subsystems being called. And even then the error message is usually quite telling as to which subsystem is broken. All the while unit tests totally fail on testing configuration issues. So needless to say, I am not more than ever convinced that unit testing application code is a waste of time but that functional tests are an absolute must!

Anyway, since we decided in my team to get more serious about functional testing we decided to make it less time consuming. First up we had to fix an issue where the FBML fb: prefix would cause our tests to fail on Debian. Thx to the Liip CEO Chregu and team member Dan for figuring out that we had to enable internal xml errors. Fellow Liip developer Lea added tools to quickly load up fixtures, which I expanded a bit, most notably I made it some adjustments so that we could speed up tests using SQLite. Taking inspiration from Thibaults work on the FOS\UserBundle I adapted his runCommand() method to make it easier to test CLI Commands. Also after half a day of fiddeling with the Security layer and trying to come up with a good way to test pages that required authentication I finally figured out that the way to go was http basic auth, for which I also added a little helper with makeClient(). Dan worked on adding HTML5 validation, so that we could easily validate entire pages or just snippets of HTML. The results are available in the HTML5WebTestCase class.

Another issue of testing the implementation itself with mocks that have assertions (i.e. ->expects() calls) is that you are basically breaking the dependency injection feature of Symfony2. For example if you do getMock('FooBar') and then later decide to rename that class, you just update the service definition and a BarBaz instance will be injected in every class instead of a FooBar one. The unit tests however have to be updated by hand which is just tedious work for not much benefit (if at all).

Maybe we need some sort of DI-Mocking? A DI-Container that created Services with MockUps injected?

Yeah. There should be a getMockService() which returns a mock builder instance. It should probably also default to setting the constructor to be disabled. That should work for most services that do not use a factory.

Just hacked something like this up.