iBlog - Ilia Alshanetsky

Yey! Yet another stable release of FUDforum is out.

A bit more of e-mail fine tuning, this time aimed at making sure all mail clients can properly parsed encoded e-mails. Updated Chinese & French translations and the English translation undergone major grammatical revisions. The poll can now be position anywhere inside the message via the use of the {POLL} tag. Stricter URL session checks etc... All users are encourages to upgrade.

A friend of mine gave me a good idea of making a wrapper around the sendfile() syscall that can be used for very quick data transfers between socket and file descriptors. This syscall in some instances can replace 3 other syscalls (seek, read, write) when you need to write a portion of 1 file into another file. Another words a very neat functionality that I could see myself using in my own applications, such as the message file compactor for FUDforum who's entire job is to seek and write data from one file to another. Further tests have shown that this syscall can also be used to create a much faster copy() that compared to PHP's current copy mechanism is about 2x faster, not a bad improvement if I do say so myself.
However, when it came to testing the code in my primary development box it just would not work, neither would any sendfile example I could find on the net that dealt with transferring data across two files. After much searching on Google, IRC and reviewing the relevant kernel sources I've discovered the problem. It appears that as for Linux kernel 2.6.X sendfile() syscall can only be used to transfer data from a file descriptor to a socket descriptor. The so called "misfeature" of allowing data transfer between two files has been removed, while I could not find any reported problems about it in 2.4.X kernels where it can work with 2 file descriptors. The frustration is further escalated by the fact that sendfile() man page does not acknowledge this little tid-bit of information and only the very recent version even documents the EINVAL error code returned, ARGHH!!!
I am still quite curios as to why this functionality was removed, since even after much searching I was not able to find the reason for it, only the fact that it was removed sometime in the 2.5.X kernel series.

There are a number of tricks that you can use to squeeze the last bit of performance from your scripts. These tricks won't make your applications much faster, but can give you that little edge in performance you may be looking for. More importantly it may give you insight into how PHP internals works allowing you to write code that can be executed in more optimal fashion by the Zend Engine. Please keep in mind that these are not the 1st optimization you should perform. There are some far easier and more performance advantageous tricks, however once those are exhausted and you don't feel like turning to C, these maybe tricks you would want to consider. So, without further ado...
Continue reading "PHP Optimization Tricks"

Ever wonder what it takes to crash PHP, well here is a quick guide . Technically speaking PHP being a high level language should not crash, but reality speaks for itself. By knowing what could make PHP crash it may be possible to implement various safety mechanisms in your PHP configuration that would prevent users from crashing your PHP. This is quite important since a crash is not only a 'bad thing' [tm] in general but can also have several adverse affect on the web server potentially creating a possibility for a local DOS (Denial of Service). So without further ado here is the current crash list:

• Stack overflow. PHP does not have any internal stack protection choosing to rely upon the system stack without any protection. This means that if you have a recursive function or a method PHP will eventually crash.
  
  function a() { a(); } a();
  
  There are 2 solutions to this problem, 1 avoid using recursive functions they are generally a bad idea anyway, and if you MUST use them implement some counter using a global variable that would prevent the function from iterating itself more then X amount of time for values of X between 500 to 1000. The other solution involves using the xdebug extension that implements protection against stack overflows by defining a limit on how deep can recursive functions go via a php.ini value. This is a better solution in hosting environments where you have no control over the scripts that are being ran on the server.
  
  
• Bug inside the pack() function. This is not a terribly common function, however it is a part of the standard PHP distribution meaning that despite it's rare usage everyone has it and is vulnreable to it.
  The nature of the crash involves making PHP to try to allocate more memory then is generally avaliable on the system, when this happens PHP will immediately exit terminating the request or if you are using the debug build dump core and crash
  
  pack("d4294967297", 2);
  
  The only solution to this problem at this time to disable the pack function via the disable_functions php.ini directive and hopefuly non of your scripts or the scripts used by your users (hosting environment) use this functionality. Like I mentioned earlier this function is pretty rarely used so this is a more or less valid solution. Another solution is to enable memory_limit directive that would set the limit on the maximum amount of memory PHP script may use and when it tries to use more PHP will nicely terminate the request and not cause the termination of the child/process used to serve the request.
  
  
• Various header parsing vulnerabilities. Through a series of extensions PHP has the ability to work with images, these extensions include exif, GD, imagemagick, ext/standard's getimagesize(). To work with images PHP often needs to allocate various buffer to store the image data. The size of these buffers is determined by parsing the image header that contains this data. By modifying this data via the use of hexeditor or generating a bogus image header you can force PHP through those image extension to try to allocate more memory then avaliable or try to allocate invalid memory value such as -1 resulting in a crash.
  
  No example given since it would make it too easy to exploit, which is especially bad since many of these functions are often used to work on external images that web users can often specify. What it means that vulnerabilities of this nature have a potential of being exploited remotely.
  
  In most cases there are sufficient checks to prevent allocation of invalid memory sizes such as -1, so most common problem would be overallocation problem identical to the one in item #2. Just about the only solution other then the disabling of these extensions (fine solution if you don't use them) is to once again use memory_limit to prevent PHP from trying to allocate more memory then it should/can.
  
  
• Stack overflow in GD library that is exposed via the imagefilltoborder() function. The internals of this function rely on a recursive function call that given a certain type of image can result in a stack overflow.
  
  You can find an example of a crash script inside the ext/gd/tests/bug27582_2.phpt test.
  
  The solution to the problem would be to rewrite the GD library function that imagefilltoborder() is a wrapper of in such a way that the recursive function algorithm is not being used. Meanwhile the only fix to the problem is to use the disable_functions directive to prevent users from accessing this function.
  
  
• Forced over-allocation inside unserialize function.
  
  $a = "a";
  unserialize(str_replace('1', 2147483647, serialize($a)));
  
  Once we can 'force' PHP to try to allocate more memory then it can resulting in a request termination. The solution is once again to use memory_limit to allow PHP to handle the situation safely.
  
  
• Refcount overflow. In ZendEngine 1 the number of references to a particular variable is limited to 65535 because they are stored inside an unsigned short. This is something fixed in ZendEgine 2 where this value is stored inside an integer with a much greater capacity. However, since stable PHP is built against ZendEngine 1 this means that any PHP 4.X is vulnreable to this problem and what's even worse there is no easy solution.
  
  $t = array(1);
  while (1) {
  $a[] = &$t;
  }
  
  The only way to prevent this problem in PHP4 is to apply the following patch and rebuild your PHP. If you are using any external Zend or PHP modules those would need to be rebuilt as well. The only time you will not be able to do this if you are using binary modules that you cannot recompile because you do not have access to their source, such as PHPA, Zend extensions, etc...
  
  
  CODE:
  Index: zend.h
  ====================================
  RCS file: /repository/Zend/Attic/zend.h,v
  retrieving revision 1.164.2.21
  diff -u -3 -p -r1.164.2.21 zend.h
  --- zend.h      16 Mar 2004 17:36:17 -0000      1.164.2.21
  +++ zend.h      14 Apr 2004 18:01:37 -0000
  @@ -263,7 +263,7 @@ struct _zval_struct {
          zvalue_value value;             /* value */
          zend_uchar type;        /* active type */
          zend_uchar is_ref;
  -       zend_ushort refcount;
  +       int refcount;
   };
  
  
  
• Since the previous crash was specific to PHP4 let's have one specific to PHP5 to keep things fair. As you may know in PHP5 a number of extensions (SQlite, Tidy, etc...) implement native OO interface, which is very convenient, but has 1 problem. Due to the destruct order of externally loadable modules loaded via dl() function (deprecated btw) the module would be freed before the objects created by that module are freed. The end result is that when PHP tries to free the object the 'free' functions are no longer avaliable (they were inside the module) and PHP crashes.
  
  dl("sqlite.so");
  $db = new SqliteDatabase("foo");
  
  The solution to this problem is to either disable dl(), people should not be using it away, needed extensions should be loaded via PHP.ini or ensure that you manually destroy any objects created by that extensions by using unset().
  
  
• PCRE library bug. While this not a PHP bug per say since the cause of the bug is the bundled pcre library (non-bundled library suffers from the same problem) it still allows a PHP script to crash the process.
  
  preg_match('/(.(?!b))*/', str_repeat("a", 10000));
  
  There is no solution to this bug at this time, it is my hope that PCRE developers will consider this problem and implement a fix for it in the next release.
  
  
• Back to the old favorite, memory allocation. This time str_repeat() function allows us to very easily make PHP try to allocate excessive amount of memory and crash or exit when it fails to do so.
  
  str_repeat("a", 10000000000);
  
  The solution once again is to use memory_limit to allow PHP to handle the situation gracefully.
  
  
• The last way is not really a crash but still a very effective nastiness a disgruntled user can do on an Apache server.
  
  shell_exec("killall -11 httpd");
  
  As you can imagine this little command will result in the termination of all running apache children other the deamon that is running as root since all Apache children run under the same privilege meaning that 1 child could kill the rest. There are several protections against this, first of all make sure that the environment variable PATH for your webserver does not contain any dangerous paths, such as the ones containing kill & killall utilities. You can also disable execution commands by enabling safe_mode or using the disable_functions directive. The last solution is to run PHP as CGI or FastCGI where the processes would run under the same user as the one executing the script. This means that PHP script will not have permissions allowing it to kill Apache children.
  
  

There are a number of other crashes in PHP, some were fixed and some may still be present. However those tend to be inside obscure extensions that most people do not have enabled and as such are very unlikely. Other crashes while may still exist are in the process of being resolved, unlike the ones listed above that are likely to be around for quite a while.

Lately it seems that the possibility of PHP developers agreeing on something is about as likely as peace in the middle-east being declared overnight.
After about a two week cease fire, following the long and bloody fight about studlyCaps aka suckyCaps, which ended with a sound defeat of the forces of sanity & reason the PHP developer community is at it again. Once more we can thank John Coggeshall for this bit of entertainment, this time centered around wondrous PHP5 feature called "Exceptions".
As you may or may not know in PHP5 it is possible to throw exceptions, which you can then catch and theoretically through this process simply error checking process and make it easier to write code. John proposed that all error messages raised by OO code would report errors by throwing exceptions rather then using the current error reporting mechanism. Technically the idea is sound good, since it would allow reduction in the number of error checks, allowing all error tracking be performed via a single try {} catch () {} block around the relevant code.
Alas theory tends to neglect practical problems and this is where the fun begins. In PHP an uncaught exception leads to a fatal error resulting in the termination of the script regardless of criticality of the error that had occurred. Which means that if you god forbid neglect to put exception throwing code inside the block, your script will break and worse yet not provide any meaningful information other then you having an uncaught exception somewhere. More over some PHP errors can be ignored in many instances, these include NOTICES, WARNINGS and PHP5's STRICT errors, by making them exceptions you can no longer ignore them and must catch the thrown exception. This is particularly annoying for database extensions where there are numerous instances where it may be safe to allow certain operations to fail. In fact certain optimizations rely on trying to execute a query and upon failure execute an alternate query thereby avoid locking the table(s) used for for the operation, which has significant performance benefits. Even outside of the database world, having exceptions thrown on every error can do more harm then good.
Let's take the Tidy extension for example, which is first extension subject to this new exception treatment. Certain operations such as setting deprecated libtidy options results in harmless warning that can be safely ignored as it does not have any affects on the generated code. However, with exception a harmless warning becomes an exception that must be handled. Consequently the script author who intends to use the simpler OO interface will now pay for that simplicity with interest by having to implement various error handlers to ensure that his script is not terminated prematurely by an uncaught exception. What's even worse that once an exception is triggered the code goes directly to the exception handler skipping any of the latter code, meaning that you have very little choice but to terminate the script execution yourself after logging the warning or error. This pretty much the usefullness of exceptions, who's original goal is to simplify the error handling code.

Yet another proof that the road to hell is paved with good intentions.

If Porsche can have an SUV, I can have a blog!

While this is not technically the very post, it is the first official online rant by your's truly. Well, that's not entirely true either, since I rant on a regular basis on internals mailing list about wide variety of topics. But let's ignore the facts for a moment, they only tend to get in the way of things.
Anyhow, on with the introduction.
I finally decided to get of my lazy ass and design a site for myself. Rather amusing, given that I've been
directly and indirectly involved in the design of hundreds of websites in the past. For the moment only the blog is online, however I will add other tidbits in the near future.

The biggest difficulty was lack of a good (short) domain. Despite having a fairly rare name, that I tend to spell differently that some of my namesakes (Ilia vs Ilya) this had proven to be quite a challenge. All common domain extensions were taken with the expiry date no where in sight and when they did approach the said date they were promptly renewed . When I turned to country based domain extensions, to my great surprise I discovered that the vast majority of possible options were too reserved. In fact, ilia.ca is owned by some fella from Vanier (Ontario Canada) who is using an e-mail address name virtually identical to mine. For some strange reason, even though his last time does not begin with an 'A' his e-mail login is iliaa@sympatico, while mine is ilia@sympatico (HA! I got my account 1st). That said I tend to use iliaa quite frequntly since ilia often tends to be taken .

After a bit of research, I've come to a conclusion that to get ilia.[extension] I'd have to resort to using some strange country code like .cn (China) or (.tw) (Taiwan) or even stranger variants such as .co.nz. Fortunately, I remembered about the newly added .ws (WebSite) domain extension, which to my great surprise was not yet taken by some domain squatting bastard (See: http://ilia.com, http://ilia.net, http://ilia.org/) and grabbed it for myself. Which is what made the page you are looking at possible.