So what do we in the PHP world do for QA'ing our applications? Just trying to get an overview of what people are doing to make sure I am still in the loop of whats happening.
So at the very basic level you have your unit tests using something like PHPUnit or phpt . Usually at least all your core components, like your db layer, mod_rewrite link generator and parser etc. should be covered with unit tests. These should be checked by code coverage analysis. I guess in the Java space they also have tools that automatically mutate your tests in order to better figure out the quality of your test cases (if the test code is mutated and the tests still pass, you aren't writing good tests).
Personally I think there is a point when you get to your actual runtime modules, where unit tests become too troublesome to write (especially if they are very dependent on time of day or session variables like user rights and session specific states). At this point you probably want to move on to using things like Selenium (which PHPUnit apparently can also leverage) or BadBoy. Pitty I have not had the chance to really make production use of either one. But I am hoping that once the QA team in Bucharest gets off the current project, they will teach the rest of us at Optaros a bit about them.
The next area in QA I see as critical is API documentation, which can be nicely generated via PHPDocumentor or Doxygen. Here again you can analyze how many classes and methods are missing documentation. Are there free tools out there that can do comment-to-code line count comparisons yet? Obviously you should also provide end user documentation. For core libraries in PEAR there are now even documentation coverage analysis scripts. Quite nifty.
Also I think its very important that applications, or more specifically their underlying frameworks, provide debugging tools. I really like the way symfony does thing here. You can use a separate front controller, that automatically enables debugging and logging. It also automatically provides a debug overlay with request specific information, like all queries that were run etc. Unfortunately there is no solution integrated yet to get the same functionality in your AJAX requests. You can also overwrite other standard configuration options when running the development front controller. Also one should ensure that debug methods are available to dump any kind of communication (web service requests etc.).
The final level in terms of tools I see is stuff like code analyzers that check CS violations, XSS or SQL injection attacks. However the most important approach to QA'ing PHP applications is a commit mailinglist and encouraging peer review. Maybe committing some bogus code that doesn't break any unit tests to check if everybody is on their toes is necessary, if you see too few comments on peoples commits. In the same context things like paired programming and help improve the quality of code.
So what am I missing? Any tool I should have mentioned, team management approach? I am sure I have since I just wrote this blog post very quickly and I do not have a check list yet. Speaking of checklist, that sounds like a good thing to have when doing QA.
1. If you intend to deploy your application on different servers an developers computers, you should probably think about deployment and installation tool like Pake in Symfony
2. It is good idea to check performance and memory consumption with Xdebug
some other stuff:
- Zend Code Analyzer ... very helpful, there are limitations, but especially when forced to review 4 peoples code: a bummer ;)
-ant ;) Especially when starting from scratch. A parameterized Ant build is a timesaver. Especially when combined with the tools you suggest ;)
- Your bugtracker: Set up some rules, like when NOT to release.
- Release Docs: Especially in "productive" Applications a overseen thing: get every line of change in the svn since the last release and write a changelog
- Different deliverables with defined processes: Bugfix, Upgrade, Release with different levels of documentation
You should mention CruiseControl for integration tests, and Jmeter for performance tests.
- ant ?!?, isn't that called phing in the php world
- PHPFIT as Framework for Integrated Test
- php simpletest
@Veysel: I just favour it because the group behind it is much larger, the product has seen more release cycles and is well tested.
I'm partial, of course, but definitely SimpleTest. Also, an automated build system such as phing can save loads of time. You can automate packaging (if it's a deployable package a la PEAR/ZF), deployment, testing, doc building, everything. I'm still working on a "phing BringWaterBottle" target, but once I've got that in place I'll rarely have to leave the desk. ;-)
