Foiled by CakePHP

Yesterday, I was all poised to launch myself over the top and be completely caught up on my Holidailies posting. That was the plan. Unfortunately, something intervened: CakePHP.

As I noted the day before, I built the new Holidailies 2008 site using the CakePHP framework.

One reason I wanted a framework was so that I had a solid structure on which to build new capabilities. As the month of Holidailies has progressed, that's what's happened. One unfortunate side effect, however, is that the complexity of the site has now grown to the point where changes may have an unpredictable result.

When the site was small and simple, I could conceptualize how every piece interconnected, and I could visualize how a change in one place may have an effect somewhere else. As a software project (such as Holidailies) gets more complex, it's increasingly difficult to visualize all the small pieces in total. Thus, there is an increasing risk that a small change may have an unexpected effect. That's how software bugs are created.

That's why testing is important. When you have a test suite you can lock in all the expected behavior of all the components and their assembled subsystems. If you have a good test suite, any change that affects the behavior of the system should be detected. Without a test suite, an errant change results in a bug.

Testing reduces the risk of bugs. Plus, it makes development more efficient, because you can refactor and change code with confidence.

Clearly, the Holidailies code base has reached the point where it needs a test suite. So I was pleased to see that CakePHP provides a testing mechanism.

Unfortunately, after three days of failed work, that hope has been shattered by frustrating problems.

The first problem is that the test capability is misdocumented. Mind you, misdocumented is worse than undocumented. When something is undocumented, you know you've got nothing. When it's misdocumented you start with a lie. Then you waste all your time working that lie, and then you get to the point where you realize you have nothing and you can't trust the documentation. Believe me, that is not a happy achievement to show for day's work.

The second problem, maybe the worse problem, is that CakePHP doesn't even pass its own test suite. That's a stunning situation for a version labeled a "release candidate".

There are some community resources, but, after three days trying, I give up. There is incomplete and conflicting information spread between the mailing list, IRC channel, and blog posts. People can't even agree on a convention for naming test fixture filenames and classes. The only thing I've discovered for certain is that everybody who tries to setup testing on CakePHP ends up frustrated, and those few who make it only do so with days of sustained effort.

So, I throw in the towel. Normally in an open source project you file a bug incident when you find a problem, but that won't help here: this problem is widely known already.

Maybe the issue will be resolved in a future version of CakePHP. (Maybe it will be called the "less broken release candidate" version.) If that happens, I'll take another look at testing support. Holidailies is committed to CakePHP, so I'd like a test suite eventually.

So, let this be a cautionary tale for anybody evaluating PHP frameworks. In the meantime, we'll soldier on. If you notice any glitchy behaviors arising on the Holidailies site, now you'll know why. Every change I make comes with increasing risk, and at some point I'm certain to roll snake eyes.

Comments

Comments have been closed for this entry.

Maybe a different PHP framework?

Maybe for next year (or for other projects) you might want to keep an eye on CodeIgniter? http://codeigniter.com/ It doesn't cost anything to download, it's stable, it's documented VERY well, it's got an active developer community around it, and they're using this framework to power the next version of their (commercially available, pay-per-license) software, so they don't leave major things like this just...hanging. (Or maybe you've already looked at this one and decided that it won't work with how you prefer to work. It has a Unit Test class to let you test individual modules, but not a full-blown test suite.)

Re: Maybe a different PHP framework?

I'm unlikely to migrate Holidailies off CakePHP, but I'm even less likely to use for another project.

I did a quick paper eval of PHP frameworks and picked CakePHP because it looked like the smallest and simplest, and I thought that was a good place to start. I didn't consider CodeIgniter. Thanks for the pointer, I will next time.

I'm bummed to hear about your

I'm bummed to hear about your Cake woes. I've never actually finished a project in Cake, so I'm extremely grateful for your posts. I've been evaluating Zend and Prado frameworks as well. I'm disappointed in the test suite issues you're having as well, that is supposed to be a core competancy of these 'agile' MVC frameworks.

Thanks again for the posts.

I'm not talking about testing

I'm not talking about testing but CodeIgniter is even more procedural than CakePHP atm. After doing my initial evaluation on different PHP frameworks, I still like many ideas that Cake has

cakePHP a True Romance Story

I have been a die hard proponent for cakePHP for the last year, and have developed several websites using it. Including my resume website at www.codemis.com. I find my self in too different positions with cake. Either I really love it, because it makes my development fast, and provides great resources to develop with less code. OR I really hate it. Whenever I try to do something new or creative, that steps out of the coding standards of cakePHP, I begin to hate coding. Documentation is minimal, I always seem to find cakePHP 1.1 documentation instead of 1.2, and since it is open source, its always being developed on and has flaws in the system. I met the coders who work on this project, and give them high praise for the work they do, but I still want to vent to.

I have tried to install the testing suite and I ended up with two different scenarios. When I just added Simpletest to the vendor folder and ran a model testing, it would randomly choose what database tables to create, and ignored the other required tables. Then it would spit an error saying I was missing a table. What makes this funny and irritating is that I have no control over table creation. That is the test fixtures job. I was just following this tutorial: http://bakery.cakephp.org/articles/view/testing-models-with-cakephp-1-2-....

The second scenario, I downloaded the testing suite from cakePHP's website and installed it. After running it, I just got an error message that stated "call to undefined function vendor()". So now I find my self scouring the web for help. I want to get this working. So I through up my hands, and wait for cakePHP 1.2 to finally become a stable release.

We didn't even bother

I wrote an entire post about hating cakePHP, and got hammered for it :) I'm glad to see at least a few other people have the similar frustrations -- at least I know I'm not completely alone in the universe.

I'm working on a large Cake-based web application at the moment. We wrote an entire custom testing framework in order to ensure that everything is well testsed, so I've never had to actually work with cake's built-in testing framework. I have another CakePHP project that started using Simpletest. Is simpletest what cake uses?