Monday, November 28. 2005
Following the “Release early, release often” mantra, PHP 5.1.1 was released today. The main causes for this release are four regressions in behavior introduces by PHP 5.1.0, which include:
The fifth reason being a refinement of a cURL open_basedir/safe_mode security fix, to improve the checks surrounding the file: wrapper handling. The new packages and win32 PECL binaries for PHP 5.1.1 can be found here: http://www.php.net/downloads.php
Now on to the long overdue rant
The initial and probably the main reason for the PHP 5.1.1 release in such a hurry was a rather interesting problem. One of the features introduced by 5.1.0 was the addition of the native “date” class. This was done for two reasons, to allow date extension’s constants to be namespaces to the class, which makes accessing them far faster and does not pollute the global class space. The other reason was to reserve the “date” class name, to indicate that applications/libraries should not implement it themselves. The eventual plan was to enable additional ext/date functionality via this class through a series of methods that have already been written. With these seemingly reasonable goals in mind after discussing the issue with Derick Rethans (author of ext/date), I’ve applied the patch to this affect in 5.1.0RC6, with the assumption that this would not break anything. This assumption held true for an entire week leading up to the final release. Even the upgrading guide (meticulously written by Steph Fox), mentioning this feature and discussed on both IRC and internals mailing list failed to provoke any kind of reaction from either the pear::date users or the developers.
The real “fun” began a couple of hours after the release when Pierre-Alain Joye posted a message titled “PHP 5.1 (Or How to break tousands of apps out there)”, whose title was certain to provoke a reaction and boy, did it ever. Within a few hours just about every frequent responder to the internals mailing list had their say. As per usual most of the traffic was something to the affect of: I don’t like it, I don’t like because developer I usually agree with does not like it, I don’t like it because I feel like being contrary, etc… A number people did make a few suggestions (those of you who did, you have my thanks), but the discussion was quickly returned to complaining, counter-complaining, and so on. If you have a few hours to kill and enjoy seeing developers bicker, here is the full thread: http://beeblex.com/lists/index.php/php.internals/20324?s=l:php.internals
While internals mailing list was engaged in an e-mail version of world war 3, the PHP bug system at http://bugs.php.net was surprisingly quiet. When something important is broken, in a near instant we have dozens of reports on the topic. In fact, only a single bug about this issue was posted and within a few minutes bogused by the user himself, who took the time to read the upgrading guide and saw the note about the date class stab. That was all, there were no further bug reports or complains on the issue in the 4 days that the 5.1.0 release was out in the wild. The only way I feel this was possible is that if the date usage statistics were greatly inflated thanks to automated downloads and do not accurately represent the number of users using it.
At this point as the 5.1 Release Master I had a few options, ignore the issue, citing the upgrade guide, non-existent bug reports, etc… Rename the date class to avoid the namespace conflict or remove it all together.
At first I was very much against doing anything, because changing the core class to be compatible with PEAR seems like an exceptionally bad decision. I, as number of other developers feel that the 1st choice of the namespace for functions & classes should be given to the programming (scripting in this case) language rather then code written in it. By allowing PEAR or any other PHP library dictate naming conventions for the language we prevent the language from choosing names that have the least amount of WTF factor. It seems very shortsighted to have a native date class called PHP_DATE or alike, so that an optional PEAR library can make use of “date” for its class name. Unfortunately this was seemingly the minority opinion and fixing the problem in PEAR didn’t get much traction, despite PEAR’s own naming conventions saying that all classes should have a prefix, oh well. The majority seemed to be focused on fixing the “problem” in PHP and would not be swayed. My first solution was to offer to rename the class, in attempt to retain the functionality (class constants), in retrospect it was a bad idea and I am glad it was not accepted. Interestingly enough by the time it was offered, the original thread had since long abandoned the “date” discussion and moved onto more interesting things, namespaces, which are still being discussed in that thread. I guess talking about dates; can only hold people’s attention for so long. After some discussions via e-mail and IRC the decision was made to remove the class by putting it inside #ifdef, and leave the date discussion to a future point.
So, what can we learn from this? First it seems that when it comes to PHP, given a sufficiently loud minority or reasonably vocal majority it is possible to force the language design to work around the applications written in it. PEAR developers are free to ignore their very own standards, not test their code against Release Candidates designed explicitly for that purpose and then look surprised and complain when things break. This is not the first time nor the last the release process has been given a bitter pill thanks to PEAR. Just today, few hours after 5.1.1 was released, Greg Beaver informed me that Win32 phar files were broken and our Win32 build master, Edin Kadribasic had to do a last minute repack.
Are there any solutions? Once possible step would be for the PEAR’s leadership (pear group) to reign in the development forcing adherence to their standards and work on addressing namespace takeovers by existing classes. Another would be to ignore the issue and wait for the appearance of namespaces in PHP 6 and hope that they magically solve the problem. Finally, PEAR 2 can be launched aimed at rewriting PEAR from scratch utilizing 5.1+ functionality resulting in a much cleaner and faster code.
Equally importantly is to have PEAR project find the time to communicate with RM during the release cycle and notify us ahead of time of any problems. Had this been done, the entire 5.1.0 fiasco could've been avoided.
PHP 5.1.1 und Hardening Patch v0.4.8 released
Das PHP-Team heute eine neue Version der OpenSource Scriptsprache PHP veröffentlicht. Die neue Version korrigiert einige wenige Fehler, wird allerdings ohne die mit PHP 5.1.0 eingeführte native Date-Klasse ausgeliefert. Diese war nach der Ve...
Weblog: Dimension 2k
Tracked: Nov 29, 05:13
Weblog: PHP-Homepage.de - News
Tracked: Nov 29, 08:06
Clash of the titans
As one can see in this excellent rant by Ilia located at http://ilia.ws/archives/95-PHP-5.1.1-Released!.html there are different views on how this should have been handled (also read the internals ML if you need even more reading material), namely why...
Tracked: Nov 29, 15:34
PEAR - Time to Refactor?
A funny thing. At this year’s PHP Conference in Frankfurt, I was speaking about refactoring PHP code. Talking about naming conventions, I urged everybody to prefix names - at least in global namespace. Currently, global namespace is the one and ...
Weblog: inside >e-novative>
Tracked: Nov 29, 17:30
PHP/5.1.0 is replaced by PHP/5.1. within 4 days!
Yes, it's true, PHP/5.1.1 released just 4 days after PHP/5.1.0! Refer to Ilia, the release mamager of PHP5.1, PHP/5.1.0 have some fatal bugs so that they have to release PHP/5.1.1The main causes for this release are four regressions in behavior introduces
Weblog: hwblog - Ho Wang
Tracked: Nov 30, 01:41
Display comments as (Linear | Threaded)
The upgrading guide that you link to does not mention the date class. Is the version on the web site stale?
For what its worth, I think that PHP should have a Date class and PEAR should change their name.
I didn't really see any discussions about this issue on the PEAR mailing lists during this whole kerfuffle. Maybe I missed it.
Given that the date class was removed the guide was updated to reflect this change. If you want the "original" guide you can find it in the UPGRADING file part of the 5.1.0 release.
The discussion I am referring to took place on internals mailing list, which is where PHP core changes are being discussed. I do not know if this issue was even raised on pear lists.
Oops, I just realized that the class is gone, and so it shouldn't be in the upgrade guide any more. I did check the online guide when I posted about this on my blog last week and I didn't see it mentioned there then.
Wow, quick reply. I just wanted to point out that while it was a big thing on the internals list it was a non-thing on the PEAR list. I watched the thing unfold on internals but chose not to add to the heat there since I had nothing to offer on the topic. I did post about it to my blog because I wanted to plug automated compatibility testing, but I tried to phrase my blog post in a way so as not to add heat to the discussion.
There is a long discussion on internals and some bugs reports in pear.
Yes PHP should have a Date class, but this is not my point.
If a code works with 5.0, I expect it to work with 5.1.
If a code works with 5.1.0RC5, I expect it to work with 5.1.0-final and not get this kind of error.
Doing such changes only to add an empty class (there is no feature in it) and some constants in it is the worst thing one can do. The whole ext/date was added to PHP using this method, private discussions, lies, and wide commits.
But the good thing is that we finally have a great release to enjoy all the great work achieved in 5.1.x.
I spare you what I think about pear::date is responsible or does not assume the changes, or other things like we do not respect our own rules. Or maybe only our rule #1:
Backward compatibility is our top priority, it should be the case for php within the same major version.
I'll take a blame for the late introduction of the class, that was a big oversight on my part. I guess a lesson for me there for the future.
Not to add fuel to the fire, but since I've restrained myself on pear-dev...
I agree that the fact that PEAR's package is called Date is shortsighted. PEAR having DB/mysql.php in it's path has also bitten me in the ass plenty. However, it's not something that PEAR developers can fix, since there is a LOT of code out there which is already depending on that class being named Date. Not to mention any other non-prefixed third party code out there.
We talked about this very issue over lunch at Frankfurt... if I remember correctly the conversation started with me saying "Are you guys serious" and ended with Derick saying "I guess we'll see". So I don't think that the PEAR community is totally to blame.
As far as the rest of the release, I found it to be one of the most solid releases I've tested. Before 5.1 rolled, I tested it with Wordpress, Joomla, CivicSpace, all of my code and a couple of other third-party apps and it work seamlessly. I'm working on some code to do PEAR testing and hopefully will be adding that to my test so that we can automate some of this stuff in the future.
They can fix it by renaming the class, yes it'll break BC but the longer the decision is made the more painful it'll be. I think you'll agree that eventually they'll have to do it.
Ilia, you know I'm a supporter of core, that's not an issue..
However, asking PEAR to break years of historical BC is like actually expecting PHP-core to switch the parameter order in the array/string libraries to "fix" them.
It's just too much of a problem in both camps.
I'm glad the 5.1.1 fix is out. I'm glad we've collectively learned that adding things _can_ break stuff (especially in a RC).
Whether or not the core developers LIKE PEAR is often an issue, but as Rasmus put it:
"people don't make the distinction. They download stuff from php.net and at the very least they expect it to not conflict"
Anyway.. not trying to take wind out of your rant. It should've been tested. We're all guilty. 5.1.1 looks very solid (-:
Of course PEAR can fix it by "renaming" the class. The path to go is:
1) Introduce a new class with a proper prefix which is
100% compatible (except for the prefix) with the old
2) Modify the old code to use this new class.
3) Remove old "Date" class from PEAR
After step 2, a reasonable amount of time should pass before doing step 3 (about 6-9 months sounds about right). Until step 3 is taken, no old code breaks, so there would be plenty of time for application developers to modify their code.
However, as far as I understand this issue, the Date class introduced in 5.1.0 and removed in 5.1.1 was empty (didn't do anything). What should it do later on? I ask because one option would be to use an implementation of the date class in the PHP core which is complatible with what the PEAR Date class did.
You give most developers too much credit.
PEAR is still compatible with PHP 4.2 because people complained when it broke. PEAR is a strange beast, and seemingly needs to be backwards compatible forever.
Yes, it's a broken system, but there's no easy fix. Hopefully namespacing will ease this burden, but it won't immediately solve it.
Namespace won't magically solve the problem because:
1) To use namespaces PHP 6.0 will be required, meaning that PEAR will need to abandon all semblence of backwards compatibility.
2) In order to benefit from namespace you need to USE THEM, just like you need to use prefixes right now to prevent naming conflicts. This means that if PEAR developer choose to retain old names for functions/classes nothing will change.
Well, it is the choice of the PEAR developers wether they respong to such complaints by reverting to the old names or by telling those complaining to update their apps.
If a class just changes its name, not its behaviour, the transition to the new name is usually (there are exceptions) very easy.
It just isn't OK for a third party developer (which I assume the author of PEAR's Date class is) to polute the original namespace of the base language. At least it is not OK when that third party developer (and the users of his class) expects the base language never to introduce a feature which clashes with the name he chose.
And especially DATE is a name which is VERY likely to be used by the base language at one time. He should have expected this all to happen some day and therefor he should have chosen to adhere to the standards the PEAR project has set up and use a prefix for his class.
In my opinion, you either accept that a programming language introduces new features at some time, which might cause some additional work for you, or you have to stick to the old version of that language.
After all, as far as I know, php4 is still supported. If you want PHP5, you might need to invest some work. But if you don't want to change your program anyhow, what do you want to upgrade to php5 for?
I agree wholeheartedly. PHP5 should move forward. While it seems to be gathering pace, PHP5 uptake so far has been slow. So despite the post-5.1 FUD, there just aren't that many people out there who need to run PEAR on PHP5.
Sven, get the point one.
The problem is not to use Date as an intern class. I was the _first_ to propose it 2-3 years ago and to propose code for that, for php5.
The problem is to add such things in RC phase, without telling anyone, and breaking apps that worked with 5.0 as well (that kills the argument about moving to php5 needs some work, iirc).
I never argued specifically about pear::date but about all the possible breaks in the userland.
The main reason that PEAR needs to remain backward compatibility is that it was historically used to "value add" the core. With improvements in core and with the great work done in PECL, PEAR is rapidly becoming pointless.
I really like PHP 5.1, and look forward to using it. I've also enjoyed very much working directly with you, Derick, and other RMs trying to iron out kinks in the PHPPEAR relationship, which has been declining for several years. This is why this part of your post irks me a bit:
"This is not the first time nor the last the release process has been given a bitter pill thanks to PEAR. Just today, few hours after 5.1.1 was released, Greg Beaver informed me that Win32 phar files were broken and our Win32 build master, Edin Kadribasic had to do a last minute repack."
It's all about communication, and I'm somewhat disappointed to be held up as the poster child of the badness of PEAR when what happened this morning was really an example of responsiveness (I woke up, was surprised to see the release, checked the go-pear.phar, saw it was broken, went on IRC, and had a fixed PHP 5.1.1-win32.zip up thanks to Edin in about 10 minutes).
I think that it would be very helpful if the PHP RM would send off a "release imminent" email to php-internals 2 days prior to any anticipated release. I had no idea PHP 5.1.1 was going to be released today, and would have easily been able to notify Edin of changes to go-pear.phar that needed to be in the release, averting this problem.
In all honesty once the problem was identified Greg's response was quite efficient and quick. It should be noted that Greg is the guy behind the pear install process, if pear-core is anything to go by, he is pretty much THE author of it.
I realize I singled him about in my rant and my apologies for that. It was just the most recent instance of PEAR/PHP miscommunication I could bring out. Through out the 4.3.X release process, which I RMed as well, there were many PEAR PHP issues, but I could not find the relevant ML entries.
As far as notification goes, I did post a message on internals about 5.1.1 being scheduled for Monday.
However, given the on going flame war on the list with over 150 messages in the past few days I cannot honestly blame him for missing it.
Maybe we need to open up the internal communication channels between PHP/PEAR guys who are involved in ensuring that PEAR is ready for PHP releases?
Also maybe integrate milestones into bugs.php.net and document the release process so that we have a checklist to go buy that has point form a list that needs to occur prior to a release going out the door?
As a PHP user I think it's unfortunate that there seems to be a chasm between PHP core development and PEAR development. It seems to me that the class- and function library is something that defines a language as much, if not more, than the language syntax itself. So the sentiment that I get about PEAR being an afterthought to PHP developers is a bit disheartening.
What defines PHP is very much it's easy syntax, readily available documentation, that is extremely easy to use and access and a plethora on native functions and extensions.
As far as classes/libraries outside of the language for PHP there are hundreds of those, PEAR is just one of them.
>Through out the 4.3.X release process, which I RMed as
> well, there were many PEAR PHP issues, but I could not >find the relevant ML entries.
I should not take that personnally but I do.
4.2.x was a pain, it was a fight to convince you (internals) to keep pear in the releases and keep it sync'ed.
But saying that the 4.3.x (and 5.0 for that matter) was problematic is not honest. We worked a lot together to fix all communication problems and keep pear udpated in each release. Andi, Edin and you (at a later point) helped us to provide the best we can in php releases.
I do not say it was always easy, some releases appeared to be done without too much noises around them, like 5.1.1 for example.
Just my $0.02, PEAR, though still widely used, is becoming increasingly irrelevant as most of it's useful functionality is subsumed by core and PECL.
John Carney & al, when we say PEAR here we are talking about the installer only.
Consider taking an example at the recently released Firefox 1.5.
Release Candidates should reflect the final programme (not even have an RC or -dev in the version) and should introduce nothing but critical bugfixes, an example would be the phar files broken on win32.
RCs should also last at least two weeks until they become official version.
Which means in return, that you should consider releasing alpha releases and "real" beta releases, instead of calling alphas betas and betas RCs.
Also, a PEAR and most importantly also PECL testsuite should be created and tested with every alpha/beta/rc to ensure that it works.
There is usually at least a week before RC goes final, which was given in this case and in most cases there is more then one RC.
Introduction of date class was a semi-critical issue because it involved reservation of namespace for PHP.
Ilia: My personal feeling, mostly as a PHP user, still is that something worsened recently with the releases of PHP.
4.4.1 introduced some annoying things (which hopefully get fixed soon), 5.0.4 had a serious readfile() flaw, 5.0.5 took ages to be released and had the reference patch in it, no 5.0.6 in sight despite of security issues (don't tell me all ISPs/people will upgrade to 5.0.5 to 5.1.x immediately) and now the date issue in 5.1.0.
It just feels wrong, as such things shouldn't really happen. Maybe this coincidence should let you draw some conclusions that despite you already do some effort in making quality releases, these efforts should be increased.
Don't get me wrong, I don't want to plainly complain, I just want to remind. I can already see some hesitations by people to try new PHP versions because they think it is like playing jeopardy and I don't think their prejudices should become truth.
With PHP6 quite a while away, you have a chance to stabilize 5 inside out. Try to make the best of it.
I agree for most parts but regarding the test suites, we already have those on PEAR it self (the installer and the core) but we lack a lot on the packages them selfs and we kinda can't force the developers to write them, we could put some rules as to that there should be basic tests before a stable release but that kind of rules just make the developer linger in beta for the longest time ... kind of a hell circle so to say :/
I think most people would recognize that the introduction of core features will sometimes clash with existing userland code.
I also think that these points need to be taken from the events of the last week:
1) Introducing new features in an RC should be forbidden.
2) You should not expect application/library authors to test every single RC release as they won't.
3) Application/Library authors should have in place a notification policy to notify their users of imminent changes to the core that will affect their code. They should also only provide assurance against tested versions of PHP.
4) Application/Library authors should bundle PEAR if they are concerned enough about the stability of their product.
I am also in favour of a PEAR2 to leverage php 5.x. Seeing as BC is so important, I would think that it would be obvious to leave the old code behind, ensuring BC to php 4.x. Maybe they could have a PEAR4 and a PEAR5, which would provide an indication into it's support for the major PHP version.
I had a problem after upgrading to PHP 5.2.0 caused by the new date functions. Specifically "date_format()" conflicted with a pre-existing date_format() function already in Phorum 3, a PHP app. Looking on the web, I also found a PHP app called Achievo (archiving) that also has the same problem.
Introducing new functions, especially internal functions like date_format breaks PHP code and discourages upgrades ;-(. Internal functions should use their own name space or at least have some prefix (such as "_").
The workaround I used was to rename the Phorum date_format function to phorum_date_format, but not everyone will want to go through all this trouble.
this is why people developing common use applications such as Phorum need to avoid creating functions/classes with common names to make their applications future proof. People need to learn to allways prefix their function/class names with their application's name to avoid problems.