CVS-Version: $Id: ChangeLog,v 1.18 2004/04/09 22:08:19 weigon Exp $
** NOTE **
This file is abondend in favour of the NEWS file
** NOTE **
15.06.2005 20:10 - 1.3.15
- mod_cml, mod_trigger_b4_dl
added both plugins
check http://trac.lighttpd.net/trac/log/branches/lighttpd-1.3.x/ for the
rest
15.06.2005 13:59 - 1.3.14
- documentation
more fix-ups for mod_fastcgi, configuration and mysql-vhost
- tests
unified test-scripts to use the same framework
14.06.2005 23:38
- documentation
updated docs for mod_proxy and mod_scgi
12.06.2005 12:55 - pre-release
- mod_fastcgi
fixed event handling after delayed connect
make retry timeout configurable
removed unused events if we switch to FCGI_STATE_READ
disabled adaptive spawning
- mod_proxy
added hash-based and round-robin balancing
use port 80 is default
fixed errorhandling on connection refused
fixed usage of dead hosts [285]
- mod_cgi
send 500 if the CGI died before we received any data
redirect the stderr of CGI to our errorlog
cleanup cgi-env after we finished our request
- uri-handling
don't replace + to space before the ?query part
accept characters > 127 again
- mod_auth
fixed segfault if debugging is enabled [371]
- mod_scgi
added SCGI support for python + WSGI [381], [382]
- mod_setenv
fixed conditionals for setenv.add-environment [109]
- generic
report an error on duplicate config-keys in configfile [267]
call initgroups to join the groups of the users [239]
fixed automatic redirects if SSL is used
fixed compilation on Solaris [283]
disable accept-filtering on FreeBSD if ssl is used [320]
fixed handling of range-request larger than 2Gb
fixed comparisions of mime-types to case-insensitive [266]
added quotation of ETags [376], [378]
06.03.2005 12:22 - 1.3.13
- mod_accesslog
fixed NULL dereference on logfile cycling if accesslog is not set
05.03.2005 17:30
- mod_proxy
handle delayed connect in the same way as mod_fastcgi
03.03.2005 19:00
- freebsd
handle remote-close gracefully
- ssl
handle remote-close gracefully
- dir-listing
added dir-listing.hide-dotfiles and added an option to set css files
02.03.2005 19:41
- fastcgi
fixed path-info for prefix-patterns
fixed path-info for PHP
02.03.2005 14:47 - 1.3.12
- mod_fastcgi
fixed handling of dead fastcgi process which sent their SIGCLD to
initd and not to lighttpd. Moving the daemonize before starting
the fastcgi procs fixes this.
01.03.2005 23:59
- irix
fixed minor compile issues with MIPSpro
01.03.2005 11:57
- request-handling
handle most duplicate headers as 400, Bug #25
- mod_fastcgi
added more checks
28.02.2005 18:26
- mod_expire
don't ignore 'modification' any more, Bug #39
only handles static files
- last-modified
really compare timestamps, Bug #34
28.02.2005 11:54
- buffer
rewrote int2buffer functions
simplified path_simplify
- pid-file
ignore EACCESS on unlink
28.02.2005 10:35
- mod_fastcgi
accept \n\n, Bug #32
- cygwin
moved functions used by plugin and main-program to *-glue.c
- lemon
removed warnings about shadowed variables
28.02.2005 01:00
- ssl
added ssl.ca-file, Bug #19
improved error-messages
27.02.2005 23:48
- test cases
rewrote to test-framework to use Perl + Test::More
ported more test-cases
22.02.2005 01:20
- mod_fastcgi
replaced inet_addr by inet_aton
- request-handling
fixed segfault if host is empty
20.02.2005 20:05
- cgi
fixed cgi.assign for empty handlers again
20.02.2005 16:54 - 1.3.11
- request handling
ignore \r\n before request-line in keep-alive requests (Bug #13)
- cgi, fastcgi
added REMOTE_PORT and SERVER_ADDR
fixed check for broken status-code in nph-scripts (Bug #14)
- fastcgi
fixed passing cmdline options the spawned binary (Bug #16)
fixed failed reconnects (Bug #9)
- macos x
fixed build on max
- mod_secdownload
parse conditional secure-download.secret (Bug #8)
- mod_userdir
redirect if username is not followed by / (Bug #15)
13.02.2005 17:33
- mod_auth
fix crash if require, realm or method are empty (Bug #5)
13.02.2005 14:52
- network
handle EPIPE and ECONNRESET as 'client has closed connection' in writev()
(Bug #1)
- macosx
compile error on MacOS X due to missing environ (patch by Johan Sörensen)
(Bug #2)
- indexfiles
append the detected indexfile only once to uri.path (reported by Thomas
Seifert)
(Bug #3)
06.02.2005 15:16 - 1.3.10
- fastcgi
display a error-message if a hostname if specified in fastcgi.server->host
we need an IP here
- debug
added debug.log-state-handling
- spawn-fcgi
accept a full commandline for spawning
06.02.2005 12:50
- fastcgi
fixed openssl handling
- network_freebsd_sendfilev
gracefull handling of connections closed on client-side
removed debug-message
06.02.2005 01:44 - 1.3.9
- documentation
added docs for SSL setup and mod_status
- fastcgi
fixed config handling on PowerPC for local-spawning
05.02.2005 15:14
- fastcgi
added bin-environment to setup the environment of the spawned process
added bin-copy-environment to copy only the specified set of options
from the old environment
added handling of cmd-line options to bin-path
- setenv
fixed crashed in setenv.add-response-header
04.02.2005 18:09
- configure
fixed docs for --with-mysql
- fastcgi
improved performance of building the header (drop strlen())
04.02.2005 01:59
- cgi
don't send file on error
check if cgi-handler exists before executing it
added support for nph-...
02.02.2005 21:18 - pre-release
- request parsing
handle invalid characters in URI
02.02.2005 15:12
- makefiles
dropped unused header files from the distribution
02.02.2005 14:18
- fastcgi
delete sockets on shutdown
- http/1.1
adding option to disable http/1.1
01.02.2005 12:03
- cygwin
fixed plugins_load to use the right extensions again
removed mmap check
added ssl support
01.02.2005 01:49
- configure
make check for valgrind.h covered by --with-valgrind
- mod_localizer, mod_maps
remove both plugins from the distribution
- file-not-found
handle file not found again
30.01.2005 16:44
- HEAD requests
don't send content on dynamic HEAD requests with status 200
30.01.2005 15:16 - 1.3.8
- network-handler
remove debug output on writev() if the remote side closed the connection
- directory index
handle EACCES correctly
29.01.2005 15:16 - pre-release
- mod_alias
fixed mod_alias + pathinfo handling
- mod_accesslog
added access-log to syslog patch from allan
28.01.2005 17:30
- directory redirect without Host-header
use server-ip instead of client-ip for the Location:
- fastcgi + pathinfo
if fastcgi-auth redirects to a directory which doesn't exist handle it
correctly (bug introduced in 1.3.8)
- requesting directories
clean physical.path if directory is requested and dir-listing is disabled
send 403 again (buf introduced in 1.3.8)
28.01.2005 12:08
- fastcgi
ignore FDEVENT_HUP for unix-sockets as a simple read + timeout will do
the job anyway
22.01.2005 20:28 - pre-release
- fastcgi
send content and headers if authorizer mode is used
use a new connection if connection is died to fastcgi
and we have not used it yet
18.01.2005 21:21 - pre-release
- plugins
added version-id to plugins to detect plugins which are not up-to-date
16.01.2005 23:11
- fastcgi
fixed write-failed after crash of fastcgi-child
16.01.2005 20:43
- setenv
fixed setenv.add-environment
- fastcgi
fixed authorizer + added testcases
16.01.2005 17:40 - pre-release
- mod_status
beautified mod_status
- mod_setenv
added setenv.add-environment
- timeouts
add timeout to read-post
15.01.2005 12:57
- debug
added debug options to log
- missing files
- request header
- response header
- request handling
added a more usefull error message for the status-code changes in the
request parser
- server announcement
set Server: header for dynamic content too
- fastcgi
fixed double free
don't crash on FDEVENT_ERR
added a comment for EAGAIN on connect()
08.01.2005 17:45
- ssl
report an error if ssl.engine is enable but no ssl support compiled in
08.01.2005 12:23
- mod_status
added request time to the output
(late changelog) added host and filename to the output (fobax)
(late changelog) HTMLalized the output (fobax)
06.01.2005 19:51 - pre-release
- error-handler
let the error-handler handle 403 requests too
make the error-handler setable by a module
- error-pages
reworked the error-page handling
05.01.2005 13:10
- keep-alive handling
made sure that keep-alive is really handled correctly
04.01.2005 17:02
- mod_setenv
added a module to added request and response headers on the fly
- error-log
send error log to syslog() if no errorlogfile is specified (again)
02.01.2005 22:44 - pre-release
- response handling
cut of body for status 301, 304 and 205
- buffer
optimized all _hex functions (Silvan Minghetti)
02.01.2005 20:32
- fastcgi
if bin-path is not specified, don't die (bug introduced in the last pre-rel)
- auth
if userfile is empty don't auth.
02.01.2005 19:06
- mod_compress
fixed off by one if cache-dir is not set
02.01.2005 16:10
- conditional config
fixed !~ and !=
- buffer
copy empty buffers correctly
31.12.2004 17:45
- ipv6 + pidfile
don't complain if we can't remove the pidfile (Silvan Minghetti)
remove ipv6 option from the commandline of lighttpd doesn't support
ipv6 (Silvan Minghetti)
31.12.2004 15:41 - pre-release
- kqueue
simplified event handling (adam)
- fastcgi
fixed div-by-zero bugs in the adaptive process spawning
- mysql-vhost
added mysql-vhost (Christer Holgersson)
30.12.2004 19:09
- fastcgi
added adaptive spawning of FastCGI processes
- traffic shaping
added traffic shaping per virtual server
28.12.2004 23:26
- traffic shaping
added traffic shaping per connection
25.12.2004 22:58
- mod_status
fixed status.url again (Timo)
21.12.2004 11:29
- configure
added check for signal and select (compile fix for netbsd 1.4 and 1.5)
11.12.2004 12:38 - 1.3.7
- fastcgi + php
retry to connect to another PHP child if one of them dies after
connect
- cgi + multipart
don't transform CONTENT_TYPE to HTTP_CONTENT_TYPE
- debian
more cleanup, updated changelog, added more deps and suggests
(Chris Brown)
10.12.2004 22:33
- event handler
fixed crashes in kqueue
10.12.2004 13:57 - pre-release
- mod_status
fixed wraparound in total requests and total traffic
- debian
updated licence and packaging
- security
call setgroups() to get rid of all groups
- ssl
handle SSL_shutdown() == 0 correctly
fixed openssl detection in configure
fixed handling of chunked encoding
- request handling
handle Connection: keep-alive correctly (case as not ignored)
21.11.2004 02:39
- windows
merged basic native windows port (compiles with mingw)
20.11.2004 18:43
- conditional
ported
- cgi
- secdownload
- expire
- localizer
- usertrack
- status
- proxy
- server-tag
Server: ... can now be specified by server.tag = "..."
- spawn-fcgi
fixed typo in usage text
- ssl
fixed detection of libs and headers
05.11.2004 16:01
- fastcgi
added more usefull error messages
04.11.2004 23:01
- ssi
added support for ${...}
03.11.2004 14:51 - 1.3.6
- fastcgi
added spawn-fcgi to the distribution
added spawn-local-fastcgi yourself ( bin-path )
03.11.2004 11:22
- accesslog
don't cycle accesslogs of external processes are used
02.11.2004 15:34
- fastcgi
handle END-OF-REQUEST correctly if chunk-encoding is not used
02.11.2004 10:53
- internal redirects
fixed handling of query strings in internal redirects for directories
02.11.2004 09:54 - pre-release
- cgi
add REMOTE_USER, suppress AUTHORIZATION
handle payloads > 4k
- mod_alias
fixed url checking
- follow-symlink
fixed config
31.10.2004 11:30 - 1.3.5
- writev
fixed seg-fault in debug-message if write() fails and LFS is enabled
handle EINTR
- sendfile linux
handle EINTR
31.10.2004 09:09
- freebsd
added missing header in joblist.c
fixed test-scripts for zsh
30.10.2004 22:26
- modules
added mod_userdir and mod_alias
added docs for the new modules
30.10.2004 19:52
- porting
added defines for MAP_FAILED for NetBSD 1.3.x
30.10.2004 18:54 - pre-release
- pipelining
fixed offset calculations
- ipv6
IPv6 might be disabled at compile-time
- rewrite
close mem-leak
- auth
forgot to reset the global-config handler
- symlink
add option to disable follow-symlink
- ssi
added support for exec-cmd
23.10.2004 - 1.3.4
- max-fds
set the upper limit of fds only if server.max-fds is set
23.10.2004 13:49
- accesslog
use a shell to handle accesslog-pipes
22.10.2004 17:00
- accesslog
added logging of user-supplied data via %{...}o and
X-LIGHTTPD-* header
22.10.2004 14:57 - pre-release
- openwrt
fixed configure-checks and Makefile.am's to build cleanly with a
cross-compiler
builds cleanly for openwrt
22.10.2004 13:03
- out-of-fd
improved the out-of-fd handler
- cgi, fastcgi
set SERVER_NAME to server.name or the value submitted by Host:
- error-handler
only set old status code if it wasn't set by a handler
21.10.2004 22:36 - pre-release
- fastcgi
don't crash on out-of-fd condition
- out-of-fd
try handle the out-of-fd condition in a sane way
21.10.2004 15:03
- mod_auth
seperated auth.backend.*.userfile for plain, htpasswd and htdigest
added 'digest-auth' against 'plain-backend'
added auth.debug for debugging
16.10.2004 10:18 - 1.3.3
- mod_simple_vhost, mod_evhost
conditional-ized
- mod_rrdtool
maintain the request-counter for each conditional-config (adam)
14.10.2004 11:30
- accesslogs
cycle all access-logs
- mod_rewrite
tell the user to install pcre.h if he wants to use mod_rewrite
10.10.2004 10:11 - pre-release
- error-handler
added a error-handler for status 404 (server.error-handler-404)
09.10.2004 16:28 - pre-release
- cgi
added support for \n in headers
- mod_auth
added conditional auth
01.10.2004 09:28
- plugins
fixed off by one error in plugin initialization (Mike)
related into a segfault on AMD64
30.09.2004 21:44 - 1.3.2
- file-cache
disabled the file-cache it was taken the wrong files from the cache
30.09.2004 08:39 - 1.3.1
- file-cache
drop a unused file-cache entry after 10 seconds
reuse unused entries
- request-parser
accept IPv6 adresses in Host header
- tests
modified the scripts to work with zsh (check on Linux, Irix and FreeBSD)
26.09.2004 12:28
- comparission function
file-cache has delivering the wrong entry if only the last character of
the filename differed and the filesize was the same.
- cgi + cygwin
cgi need s SYSTEMROOT environment
22.09.2004 08:55
- network
detect of file has been shrinked while we are sending it out and terminate
the connection if would run over the edge
22.09.2004 07:56
- mod rewrite, fastcgi, ...
keep REQUEST_URI after rewrite
21.09.2004 22:49
- fastcgi authorizer
fixed cleanup code (matt)
21.09.2004 20:08
- rrdtool
rrdtool.db-name is now conditional
fixed check if write() failed (adam)
17.09.2004 17:50 - 1.3.0
- rewrite
added url.rewrite-final = ...
17.09.2004 15:55
- code cleanup
integrated the fixes from cygwin into the main tree
- kqueue
init kqueue after daemonizing (broken since 12.09.2004 14:02)
16.09.2004 21:00
- cygwin + macosx
finished the cygwin port
this port seems to fix the problems on macosx too
12.09.2004 14:02
- socket handling
added support to handle more than one server socket
11.09.2004 12:23 - 1.2.8
- EINTR
handle EINTR for linux-sendfile
- configfile
ignore an extra comma at the end of the array declaration
11.09.2004 09:46
- mod_proxy
pass remote-addr as X-Forwarded-For to the real server behind the proxy
- code cleanup
moved all cut'n'paste versions of the inet_ntop cache to inet_ntop_cache.c
- fcgi
don't overwrite the fd in fcgi_establish connection if connect fails. this
results in various problem in other places.
05.09.2004 09:46
- file-cache
cache the mimetype
- last-modified
don't complain if the If-Modified-Since contains a valid comment/option
like <timestamp>; length = ...
05.09.2004 09:13
- expires
overwrite the Expire if it is set by a previous plugin
- conditional config
conditional config as disabled in 1.2.7 by accident
04.09.2004 10:02 - 1.2.7
- mod-proxy
remove the \0 before the post content
- cgi
fixed hanging process if cgi-crash terminates to fast (before we read its
response)
- extented attributes
added xattr support, submitted by Ari
29.08.2004 16:00
- rrdtool
moved the rrdtool support from mod_status into its own module mod_rrdtool
rrdtool.binary = "/usr/bin/rrdtool"
rrdtool.db-name = "/var/www/lighttpd.rrd"
29.08.2004 11:00 - pre-release
- timeouts
server.max-keep-alive-requests = 0 replaces
server.use-keep-alive = "disable"
added
server.max-keep-alive-idle
server.max-read-idle
server.max-write-idle
- docs
added a entry for each config-value into configuration.txt
added simple docs for
rewrite
redirect
compress
cgi
simple-vhost
29.08.2004 10:05
- config options
complain if no configfile is specified
- fastcgi
removed stupid allocation bug which might cause a problem in really rare
cases
26.08.2004 22:06 - 1.2.6
- optimize
use array_strcasecmp() in favour of strcasecmp() as it is slightly
faster.
apply the case-insentive conversion also on the last character. (adam)
sort the checked elements in request.c and filter apply the logic to
compare some less fields, if the header is not used.
improved the config-patch function to use our internal buffer-compare
functions instead of strcmp
22.08.2004 16:09 - pre-release
- cgi
added missing cleanup code
- fastcgi
remove double-free
added handling of EINTR in some places
- leaks
fixed some leaks in the new config code
- array_strcasecmp
fixed alignment in the improved array_strcasecmp function (adam)
20.08.2004 14:46 - pre-release
- performance
optimized a few useless strlen() away as we either know the length from
buffer->used - 1 or by sizeof(str) - 1 if it is constant.
optimized the 'find the \r\n\r\n' function.
improved the array_strcasecmp() based on another idea from (ralf)
- accesslog
enabled the strftime cache again
15.08.2004 23:41
- accesslog
added apache-like CustomLog handling in accesslog.format
accesslog.format = "..."
15.08.2004 21:08
- test-cases
remove testdir
- configfiles
handle escaping of " in strings properly
13.08.2004 12:07
- array
improved inner-loop of array_strcasecmp() (ralf)
11.08.2004 14:14
- fcgi socket
use SUN_LEN if available
- keep-alive
disable keep-alive on request
server.use-keep-alive = "disable"
10.08.2004 15:59 - 1.2.5
- conditional config
mod_fastcgi
mod_rewrite
mod_redirect
mod_access
mod_compress
mod_accesslog
are ported
10.08.2004 13:05
- pipelining
fixed very stupid pipelining bug
09.08.2004 22:07 - pre-release
- conditional config
first code for conditional config
09.08.2004 14:21
- fcgi
fixed access to free()'d memory (doesn't create any harm)
- isdigit, warings
signed -> unsigned for 2 more isdigit() calls (adam)
removed some unused var's if pcre is not available (adam)
08.08.2004 20:57 - pre-release
- debian
added a chmod to /var/log/lighttpd/ (allan)
08.08.2004 12:05
- kqueue
use EV_SET() instead of setting the ev-struct by hand (adam)
- fcgi
fixed the EINPROGRESS handling to use getsockopt (er)
fixed a leak of server is disabled (er)
- solaris 10 port-api
added a skeleton for the sol10 port api
06.08.2004 10:18
- mod_ssi
fix DATE_LOCAL so it displays the correct time zone (Jeremy Hinegardner)
04.08.2004 11:43
- openbsd fixes
dropped usage of MAX() in buffer.c
added prober includes for md5.h if openssl is enabled (brad)
- memory usage
documented the way how lighttpd caches memory blocks
reset the buffers after they have been written by the network-layer
- kqueue
modify fd-bitmask only if kevent succeeded (adam)
03.08.2004 15:09
- mod_compress
compress even if you have no cachedir set
03.08.2004 13:26 - pre-release
- Makefile
fixed dependencies for parallel build in mod_ssi_expr.c
- combo patch
* Tinker with kqueue(). Add a reset method so that the kqueue file
descriptor can be re-enabled after a fork(). Emulate the devpoll driver
in that adds and deletes are sent to the notification mechanism im-
mediately, which should cut down on phantom events. Use
ev->kq_results as a sliding window.
* Change F_SETFD calls to use the preferred FD_CLOEXEC instead of 1.
* Remove unnecessary fdevent fcntl handlers. It appears that the only
driver that needs one is Linux RT signals.
* Quiet compiler warning about unused parameter.
* Set the close-on-exec flag for the /dev/poll and epoll_create() file
descriptors.
* Return failure if /dev/poll could not be opened instead of logging
and continuing.
* Detect EAGAIN after writev() failures. FreeBSD sendfile() doesn't need
protection, as the man page says:
When using a socket marked for non-blocking I/O, sendfile() may send
fewer bytes than requested. In this case, the number of bytes success-
fully written is returned in *sbytes (if specified), and the error
EAGAIN is returned.
(adam, georg, matt)
02.08.2004 18:08
- mod_ssi
check for pcre before compiling the module
- fdevents
dropped fdevent_fcntl added by the last patch (adam)
kqueue: events == FDEVENT_IN -> events & FDEVENT_IN (adam)
31.07.2004 22:07 - 1.2.4
- fdevents
* Test at configure time for kqueue() and <sys/event.h>
* Remove various hard-coded constants from event handlers
* Move maxfds into the event structure, and out of the
fdevent_*_init handlers. Event handlers can use the maxfds
member to size arrays.
* Various event structure renames to discourage clashes
* Remove extra (ignored) call to fdevent_event_next_fdndx() in
the main server loop.
* Wrestle with kqueue(). The implementation has to deal with
phantom events (for fds which have been deleted/closed), similar
to the Linux RT signals code. Like the RT code, it maintains a
bitmask of active fds. After a successful call to kevent(), the
code will compress/overwrite dead events. The other annoyance is
that the handler must track the event filter for each fd, even
though you cannot support both read and write filters for the same
fd in one kqueue. The handler maintains a separate bitmask for fd
filters (1 == EVFILT_IN, 0 == EVFILT_OUT).
(adam)
- server side includes
added native server-side includes based on the docs from apache:
http://httpd.apache.org/docs/mod/mod_include.html
not supported are:
- exec
- nested virtual
- config.errmsg
- echo.encoding
24.07.2004
- fdevents
added a bitset to figure out if we received a event for an unregistered fd
in rt-signal (adam)
- kqueue
added kqueue support (Matt Levine)
13.07.2004 08:58
- configfile
parse keys correctly that contain a digit (Geoff Adams)
- fcgi
fixed large post uploads (Geoff Adams)
fixed uri if docroot is set (Geoff Adams)
03.07.2004 22:50 - 1.2.3
- index-files
rewrite uri.path to the index-file instead of keeping it at .../
this fixes index-file handling in FastCGI/CGI docroot is used
- close-on-exec
enable close-on-exec handling to simplify FD handling in CGI code
- cgi
keep error-log-fd open to catch the error handling for execve()
report error if cgi-exec file doesn't exist
- proxy
pass page-content on error to the user (E.R.)
code cleanup (E.R.)
- ssi
first skeleton of a plugin for ServerSideIncludes
- security
limit the headerlength again to 64k max
03.07.2004 14:23
- configure
fixed compile-check for libpcre if pcre-config doesn't point to /usr/lib
02.07.2004 18:17
- buffers
always allocate a multiply of 64bytes. this should reduce the number of
realloc()s and still doesn't has a too high overhead.
02.07.2004 11:07
- fds
connect stdin, stdout and stderr to /dev/null instead of just closing it
use dup2() instead of dup()
- accesslog
if accesslog.filename starts with a | spawn a process which will get data
in one chunk once in a while
01.07.2004 11:00
- sample config
added text/css and text/javascript mimetypes
28.06.2004 12:18
- proxy module
added a proxy module (based on the fastcgi module) and added some
documentation
25.06.2004 21:41
- last-modified handling
replaced %Z by GMT. otherwise the last-modified check will most often fail.
24.06.2004 20:20
- relax http-parser
don't reply 400 in case of an empty header-field, just ignore it
23.06.2004 22:10
- file-cache
don't cache mmap() for files larger than 64k as we run out of RAM otherwise
too fast (check with some 200mb files)
- 64bit fixes
fixed buffer_equal on sparc64
15.06.2004 19:09 - 1.2.2
- mmap cache
fixed mmap-caching in network_write.c and network_writev.c after a direct
hint by E.Rodichev
- sendfile + linux
check at config-time if sendfile() works on Linux
11.06.2004 15:09
- fcgi + unix sockets
added support for unix domain sockets (spawn-fcgi 1.2.0 required)
10.06.2004 11:49
- configure
use pcre-config to determine the position of the pcre headers
05.06.2004 22:06
- filehandle-cache
remove mmap-segment if filecache gets invalidated
30.05.2004 14:13 - lighttpd 1.2.1
- response headers
request headers that appear twice are grouped together like expected by
the CGI spec (concat with a ", ")
response headers behaved the same way but are not grouped anymore. They
stay seperated. Actually they are concated by \r\n<key>: <value> which is
the same in the end.
- file uploads
the handling of longer post requests is fixed now.
28.05.2004 09:13
- cgi
added support for direct calls to cgi-binary
22.05.2004 21:58
- pipeling
the code cleanup is finished successfully. Now all cases of pipelining are
handle the right way. POST pipelining was still not working up to now.
22.05.2004 12:55
- code cleanup
use the well-tested code from the write-queue as the base for the
read-queues and simplify the pipeline handling alot that way.
20.05.2004 15:08
- network backends
enabled sendfile support again (__FreeBSD__ instead of __freebsd__)
added a mmap cache as part of the filedescriptor cache
added AIX specific send_file() support (untested)
20.05.2004 10:40
- segfaults
fixed some minor segfaults on startup when no config file is used.
17.05.2004 10:58 - lighttpd 1.2.0
- documentation
reformated the documentation the doc/ directory
15.05.2004 14:45
- localizer
fixed build of localizer extension
15.05.2004 12:35
- POST requests
there is no need to die if we spot a simple POST request on a static file
- pipelining
fixed HTTP/1.1 pipelining which caused the problems with opera
- array handling
how did that bug survive such a long time ? a wrong compare function was
used in one case, but not the other.
15.05.2004 03:20
- secure and fast downloads
added a module which allows secure and fast downloading of files:
1. the application (.php, ...) controls the access to the files
2. the webserver is handling the transfer (and check the app generated
tokens)
the token is based on
- a secret
- a timestamp
- the filename
this means even if the token is is distributed by the user it will get
invalid after a given timeout (default 60 seconds)
- errorfiles
check for errorfiles before using them
- code cleanup
applied cleanup fixes from adam
14.05.2004 18:47
- fdevent handling
added a more generic callback interface to the fdevent structures which
simplifies the writing of plugins. this might destabilize lighttpd for a
while
- cgi
fixed header parsing of the header is sent in chunks and the terminator is
sent in a single chunk
- EINTR
fixed some occurences of EINTR which read()
03.05.2004 23:55
- portability
E.R.:
* portability fixes for Solaris 2.5
02.05.2004 10:15
- Expect: Handling
added incomplete support for Expect: 100-continue (RFC 2616 8.2.3) by
sending always 417 for every Expect-request (see 14.20)
we have been blamed for not supporting it:
http://lists.w3.org/Archives/Public/ietf-http-wg/2004JanMar/0059.html
29.04.2004 23:07 - 1.1.9
- usertracking
added a very basic usertracking cookie handler
29.04.2004 19:37
- network-writev
Adam:
* call munmap() on error in write()
- docs
E.R:
* documented 'check-local'
- test-env
made the shell scripts more portable (checked with zsh, bash and ksh)
fixed compilation on NetBSD
28.04.2004 22:22
- FastCGI
E.Rodichev:
* added "Authorizer" mode for FastCGI
27.04.2004 18:08
- ssl
Alexey Slynko:
* handle SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE in SSL_write the
right way.
- FastCGI
add 'check-local' instead of the implicit 'if-docroot-is-set' handling
implemented at 24.04.2004 14:34
E.Rodichev:
* remove useless extra-/ in before uri.path
* add 'prefix' notation for FastCGI processes
26.04.2004 16:52
- code cleanup
patches from Alexey Slynko:
* remove the pidfile if lighttpd terminates the normal way (if not in chroot)
* init SSL before getting daemonized
25.04.2004 19.15
- mem-leak
fixed mem-leak on broken HTTP-headers
- FastCGI
patches from E.Rodichev:
1. CONTENT_LENGTH
"If no data are attached, then this metavariable is either NULL or not
defined".
2. QUERY_STRING
If the Script-URI does not include a query component, the QUERY_STRING
metavariable MUST be defined as an empty string ("").
3. Doubling of REMOTE_ADDR removed.
patches from adam:
* fixed bug in the sizeof() patches from yesterday.
* some more *_long to *_off_t
24.04.2004 14:34
- FastCGI
don't check for localfile if 'docroot' for a FastCGI host is specified.
24.04.2004 12:13
- POST
fixed POST request handling
- chunk-encoding
the generated HEX strings where broken since the 1.1.8
(this effected all HTTP/1.1 requests without Content-Length like FastCGI-PHP)
- code cleanup
patches from adam:
* malloc + memset -> calloc
* sizeof(int) -> sizeof(<variable>)
* assign fd_set instead of memcpy()
* init fd -> connection fd pointers to -1
16.04.2004 08:48 - 1.1.8
- code cleanup
don't reuse buffer > 64k (see settings.h)
added server.max-request-size to limit the maximum request-body size
(in kBytes)
don't accept HTTP-request headers larger then 32kBytes (see settings.h)
minor speed improvements in the request-parser
More cleanup patches from adam:
* change pre-ANSI C/valid C++ syntax for function declarations/definitions
from using () to (void). Ex: int foo(); --> int foo(void);
* use static linkage as much as possible, to limit possible symbol
collisions
* whack more unneeded variables
* try and prevent any errno clobbering by storing the old errno value before
any subsequent system calls, and restoring before function exit.
* change printf syntax for unsigned variables from %d to %u
15.04.2004 18:41
- code cleanup
handle all int != size_t cases in fcgi.c correctly
check headerfields to have a value
handle both EINVAL cases of writev() before the can occur
limit content-length to SSIZE_MAX
disallow negative content-length
the usage of ltostr() has been reduced to the minimum in favour of
buffer_/append|copy)_(long|off_t)
dropped ultostr() and ultohex() in favour of buffer_*
15.04.2004 16:35
- portablity
more patches from adam:
* remove warnings for unused parameters and variables
* remove warnings for mismatched pointer assignments
* change "gtime_r" to "gmtime_r"
13.04.2004 20:48
- test-cases
made the 'make check' target self-contained
In our case we have to call the targets in the following order:
$ ./configure ...
$ make
$ make install
$ make check
because the path to the plugins is hardcoded in the binary itself
using
$ ./configure --prefix=/tmp/lighttpd-1.1.x/
...
will help if you don't really want to install before testing.
13.04.2004 00:05
- portability
adam sent another patchset:
* Wrap PCRE-specific data member access with an #ifdef
* Add const to pointer using return value from dlerror()
* Explicitly initialize pointer in the lemon parser to 0,
in order to catch missing else { } clause
* Use a time_t rather than an int for gmtime() call. On some
systems (including 32- and 64-bit SPARC) time_t is a long.
12.04.2004 17:00 - 1.1.7
- fastcgi
strip WS after HTTP-response headers coming from the FastCGI process
added REMOTE_USER to the Server->FastCGI headers
removed HTTP_AUTHORIZATION from the Server->FastCGI headers
12.04.2004 10:24
- cgi
if we don't get a partial HTTP-response-header send the content out as soon
as the cgi script is finished
12.04.2004 01:23
- compression
added bzip2 compression (supported by w3m)
12.04.2004 00:12
- configfile
add some usefull error messages if the tokenizer or the parser fail to
read the configfile
11.04.2004 22:04
- configure
added --with-ldap and --disable-lfs to the configure options
11.04.2004 20:28
- 64bit offset size
disable linux-sendfile support for linux 2.4.x for now as it don't
support 64bit transfers
fixed all assignments on the path from the stat() to the Content-Length
HTTP-header
- head requests
set content-length in HEAD requests
- accesslog
write accesslog entry on network error
write the correct amount of byte written to the accesslog
11.04.2004 11:48
- code cleanup
moved the config for the cgi-plugin from config.c to the plugin.
moved some buffers which were only used by a one or two plugin from
the server-structure to the plugins
keeping the plugins independent from the server-core is a 'good thing'
10.04.2004 19:06
- configfile parser
removed the leaks from the configfile parser
09.04.2004 23:15 - 1.1.6
- stricter http-parser
added line-folding although noone really seems to use it.
09.04.2004 18:42
- configfile parser
the hand-written configfile parser has been replaced by a LALR(1) one.
'lemon' from the sqlite guys has been used to generate the parser.
- by-host, by-url, by-file, by ...
$HTTP["url"] =~ "~$" {
access.deny = "all"
}
$HTTP["host"] =~ "." {
simple-vhost.server-root = "/home/weigon/wwwroot/servers/"
simple-vhost.default-host = "grisu.home.kneschke.de"
simple-vhost.document-root = "pages"
}
$HTTP["host"] == "incremental.home.kneschke.de" {
server.docroot = "/hasdasd"
}
at least the parser can handle it now. Currently there is no real support
for this context-based config-option. But the syntax and the parser are
done.
09.04.2004 10:58
- ssl support
enable ssl support again
- mmap
enabled mmap+write again
08.04.2004 12:34
- stricter http-parser
based on a thread at
http://lists.w3.org/Archives/Public/ietf-http-wg/2004JanMar/0050.html
the HTTP-parser has been adjusted to be more correct when it comes to
request-header fieldnames
the whitespace handling around the ':' has been relaxed as requested
07.04.2004 17:06
- sigaction
use sigaction instead of signal if possible
07.04.2004 13:55
- accesslog
use localtime-timestamps in accesslogs if struct tm has a tm_gmtoff field
07.04.2004 10:41 - 1.1.5
- -D_REENTRANT
solaris + localtime_r() needs it
07.04.2004 02:54
- mod_auth + ldap
added a ldap backend to the auth plugin.
06.04.2004 13:37
- pidfile
fixed the permissions of the pidfile (Matthijs van der Klip)
- specfile
merge the RedHat and SuSE specfile with Matthijs
- 64bit file-offsets
moved the FILE_OFFSET_BITS settings from the config.h the Makefile to enable
64bit offsets the right way
06.04.2004 12:32
- mod_expire
added an apache compatible mod_expire which adds Expires: headers to the
request
expire.url = ( "/buggy/" => "access 2 hours" )
05.04.2004 22:34
- solaris devpoll
forgot to provide the infrastructure to actually enable the devpoll event
handler. the same has been done for the freebsd-kqueue handler (which
doesn't work yet)
fixed the devpoll support while testing it on a real solaris box
05.04.2004 09:49
- debian
added debian packaging support written by Vincent Wagelaar
- solaris
Another set of patches for Solaris from Adam
* Detect <sys/devpoll.h>
* Detect and include <sys/filio.h> for definition of FIONREAD
* Detect and link against the library exporting hstrerror
* Correct typo in SENDFILE_LIB
* Use__sun instead of __solaris__ for detecting operating system.
Please see http://predef.sourceforge.net/preos.html for more
detail
* Explicitly cast arguments to isalpha() and toupper() to unsigned
char. The man page says that the functions support the range of
an unsigned char, and EOF.
* Include <limits.h> and define UIO_MAXIOV as IOV_MAX for Solaris.
04.04.2004 18:05 - 1.1.4
- pidfile
added pidfile writing after deamonizing
04.04.2004 01:05
- fdevent
added framework for freebsd_kqueue and solaris_devpoll
the solaris_devpoll one might event work (untested)
03.04.2004 16:41
- network
added framework for filebased chunks
- read-write + mmap-write
- linux-sendfile
- freebsd-sendfile
- solaris-sendfilev (untested)
and memorybased chunks
- write
- writev
made TCP_CORK a 'global' flag around the write_chunkqueue-calls
the writev() support should improve the performance for all non-static
pages.
170 req/s against 158 req/s for the following script if writev() is used
instead of write()
<?php
for ($i = 0; $i < 1000; $i++) {
print $i."<br />\n";
flush();
}
?>
28.03.2004 13:42
- cleanup
applied some cleanup patches submitted by Adam:
* variables modified in signal handlers should be sig_atomic_t
* assert statements should not have side effects
* STD{IN,OUT,ERR}_FILENO preferred instead of {0,1,2}
* dieing --> dying
* SEGFAULT calls abort directly, instead of derefencing a NULL pointer
- mod_accesslog
modified the accesslog format to be CLF compatible
set locale for LC_TIME to C
26.03.2004 16:13
- path info
enabled the pathinfo code again
25.03.2004 13:30 - 1.1.3
- portability
compiles fine now without any patches on IRIX
- hostname detection
reworked "get the hostname for HTTP/1.0 requests which don't specify a
Host: ..." to only query the name in this single case for the server side
of the connection
- errorlog handling
stderr is only used until the errorlog is init'ed
if no error-log is specified, syslog() is used
if cycling error-log fails, syslog() is used
- accesslog cycling
don't fall back to stdout anymore
- event-handler
use poll() as the default event-handler again
24.03.2004 01:37 - 1.1.2a
- error messages
added some hints to the failing error-messages
22.03.2004 01:58 - 1.1.2
- configure
some protability changes to get the 'inline' working with the MIPS CC
21.03.2004 22:00
- mod_rewrite, mod_redirect
reading the config for those two plugins was not working
- fdevents
changed the compile time setting for the event handling into a run-time
setting.
server.event-handler = "select" # poll, linux-rtsig, linux-sysepoll
added sys_epoll() for linux 2.6
select - all systems
poll - Linux 2.1.23+, all XPG4-UNIX
sigio - linux 2.4.0+
sysepoll - linux 2.5.66+
1000-4k-nok 1000-100k-nok
select 1776.99 296.52
poll 678.02 607.28
sigio 3754.46 1411.23
sysepoll 3817.67 1431.02
21.03.2004 00:10
- configure script
rewrote large parts of the lib/header detection of the configure script
20.03.2004 01:39
- fastcgi
as the docroot on external hosts might be different than the webserver
docroot it can now be specified in the config:
fastcgi.server = ( ".php" =>
( "grisu" =>
(
"host" => "192.168.2.41",
"docroot" => "/home/jan/servers/",
"port" => 1026
)
)
)
a huge internal cleanup in the config handling made the code more readable.
some more warnings and error checking should track most of the config
errors for the fastcgi plugin
19.03.2004 12:34
- external patches
Matthijs van der Klip submitted three nice patches:
- turn off writing in mod_status if status.rrd-reports is set to disable
- fix for a debug message
- get the hostname for HTTP/1.0 requests which don't specify a Host: ...
- rc-script for RedHat
- documentation
added a documentation section about authentification
- doc/authentification.txt
19.03.2004 05:11
- optimizations
implemented special versions of
- strcasecmp (array_strcasecmp)
- isdigit, isalpha, isalnum (light_*)
added a faster check for a finished header
after disabling all modules it is still possible to get 20 kreq/s.
15.03.2004 19:45 - 1.1.1
- localizer server
added the localizer-server module to the code
- chunked-encoding
Apple's Safari or HTTP-Handler doesn't handle chunked-extensions defined by
RFC 2616 correctly and doesn't ignore them. Disabled the chunked-extension
which were just used for debugging purposes.
07.03.2004 12:20
- optimization
moved the checks if a plugins support a given function from the dispatcher
(plugins_call) to plugins_call_init() to do the check only once.
equalized the plugins functions to only two types:
SERVER_FUNC()
CONNECTION_FUNC()
replaced all handwritten plugin_call-handlers with macros
made all plugin-functions 'static'
with all plugins loaded we are at 15kreq/s gain
without any plugin loaded at 16kreq/s
before the change we were at 13kreq/s
06.03.2004 21:13
- compilation fixes
fixed some warnings on FreeBSD and NetBSD by adding
#include <netinet/in.h>
ifdef'ed a pcre-entry in base.h
remove #define _XOPEN_SOURCE from http_auth.c for IRIX
crypt() on Linux needs _XOPEN_SOURCE
06.03.2004 19:18 - 1.1.0
- authentification
added htpasswd and htdigest backends to work against user-files generated
by htpasswd and htdigest.
for basic auth:
- plain
- htpasswd (crypt only)
- htdigest
for digest auth:
- plain
- htdigest
06.03.2004 17:35
- authentification
check the method in the authorization header againt the configured method
06.03.2004 14:54
- hostname parsing
added a RFC-2396 conforming "Host:" parser
added 17 checks for the parser
06.03.2004 10:25
- configuration
added a warning for unknown configuration variables.
dropped the 'specific-for.url' ideas for now as it is not known yet how to
implement it the right way
renamed some config-variables to reflect there actual meaning
- server.host -> server.bind
- server.virtual-* -> simple-vhost.*
- server.userid -> server.username
- server.groupid -> server.groupname
- server.docroot -> server.document-root
05.03.2004 15:36
- fastcgi
internals: moved all mod_fastcgi settings from the global struct to the
plugin itself
- vhosting
got a patch for enhanced virtualhosting from christian kruse -> mod_evhost
moved the virtual hosting config (server.virtual-*) to it's own module
called mod_simple_vhost
23.02.2004 10:06
- configfile
rearragned the config-file structure again to be able to add settings for
a given URL, Host, Filename, ...
This change affects some config-options directly (access.deny,
url.rewrite, url.redirect, virtual-hosting, ...)
added 'specific-for.url' for url specific config settings
- digest auth
FIX: md5-sess type
seperate the auth-backend stuff
13.02.2004 22:23 - lighttpd 1.0.3
- content-length + POST
FIX: If header and content didn't fit into one single packet the rest of
the content was not read correctly
13.02.2004 01:07
- content-length + POST
the check for content-length on a POST request vanished somehow in one of
the previous releases.
- header search
FIX: the search for header fields was up to now case-sensitive. Now it is
like required by the standard case-in-sensitive.
- browser bugs
w3m 0.2.5 adds an additional \r\n at the end of the POST requests which is
handled now
10.02.2004 10:12
- start script
took the suse rc-skeleton which states to be LSB compliant and modified it
for lighttpd needs
09.02.2004 11:48
- HEAD requests
FIX: HEAD requests for static files delived the content from the GET
request. (test case is added)
08.02.2004 15:53
- directory listings
FIX: the pathname has not encoded at all before it was transfered to the
browser. a proper url and html-encoding has been added.
added modification-time and filesize to the output
added a basic css for the virtual listings
07.02.2004 22:15 - lighttpd 1.0.2
- sample configfile
rearranged the config-file to have all the important options at the top
- docs
added a mod-rewrite options
- mod_accesslog
stdout is no longer used a default for the accesslog
- error-messages
the 404 is now reported in the error-log
07.02.2004 17:30
- configfile handling
if a key is used twice like
url.rewrite = ( "url1" => "url")
url.rewrite = ( "url2" => "url")
you get an error now. You have to write:
url.rewrite = ( "url1" => "url",
"url2" => "url")
31.01.2004 12:22 - lighttpd 1.0.1
- log cycling
added a sighup-handler to the plugin interface and fixed the log-cycling
for access-logs
- portability
disabled the interval-time optimization on IRIX
- documentation
added a lot of new documentation to README
31.01.2004 10:59
- status module
added two new options rrd-dir and rrd-reports:
rrd-reports = (boolean) # enables RRD-reports
rrd-dir = (string) # path for the daily status-files
fixed the "status-files in /"-bug this way
22.01.2004 13:38 - lighttpd 1.0.0
- simple docroot
fixed handling of docroots if virtual-host is disabled
27.12.2003 11:12
- license handling
added the first interface to license handling.
25.12.2003 23:48
- protability
Verified again that the code compiles and runs cleanly on Linux, FreeBSD,
NetBSD and IRIX
compiling with gcc and the option -pedantic works fine
compiling with mipspro cc works fine, too
- tests
added some more tests: 39 tests
25.12.2003 16:01
- protability
some compile fixes for FreeBSD have been applied and a new switch has been
added to choose between IPv4 and IPv6 on FreeBSD.
(cmdline: -6, configfile: server.use-ipv6)
- packaging
cleaned up the specfile for building RPMs
21.12.2003 01:00
- authorization
brought basic and digest auth back to life. this module as the last one
which had to be updated after the config-file changes
- test harness framework
add 3 tests for basic auth
20.12.2003 22:10
- compression
added gzip compression (gzip-header + deflate + crc)
- test harness framework
added a 22 tests to verify the correct behaviour of lighttpd
- request parsing
GET http://www.yahoo.com/ HTTP/1.0
is handled now as
GET / HTTP/1.0
- plugins
moved FastCGI and CGI handling into modules which can be loaded at run-time
17.12.2003 13:18
- compression
the directory structure is now build automaticly
15.12.2003 01:00
- compression
added a compression cache to the compression module (mod_compress)
Vary: Accept-Encoding is set now as it influences to delivered stream of
bytes. This is important for caches.
10.12.2003 00:24
- config files
a new config-file format is ready for the final release of lighttpd.
it supports:
server.docroot = "string"
server.host = integer
server.modules = ( "string", "string" )
server.mimetypes = ( "key" => "value" )
server.complex = ( "key" => ( "string", integer ),
"string",
integer )
the syntax should look familar to all who worked with Perl or PHP.
config-file handling has been seperated into a parser and a lexer. Both
are currently written by hand and will propably be rewritten into bison +
flex if time permits. But that would be a cosmetical change.
05.12.2003 03:07
- status-page
improved the status page to display
- the current connection-stati
- average throughput over 5 seconds
(requests/s and output-bound traffic)
now you can see what is going on in the server
- access-log
the accesslog is now a module and can be disabled by just removing the
plugin from the list of loaded plugins
04.12.2003 16:18
- chroot-ing
how-to: using chroot
- chroot to /home/www/
- docroot at /servers/<hostname>/pages/
- defaulthost www.example.org
e.g. (external view)
/home/www/servers/www.example.org/pages/index.html
(in chroot)
/servers/www.example.org/pages/index.html
config:
chroot /home/www/
userid wwwrun
groupid nogroup
virtual-server-root /servers/
virtual-server-default-host www.example.org
virtual-server-docroot /pages/
The FastCGI process is living outside this chroot definition as it is
started seperatly.
01.12.2003 02:06
- cleanup
in preparation for the first stable release some internals had to be
cleaned up. Basicly it was a cleanup of workflow of the creation of the
response-header. All modules can use a clean interface for this purpose
now. This is espacially usefull for all modules which have to pass some
HTTP-headers to the client.
29.11.2003 22:22
- modules
finally moved the modules to shared libraries and cleaned up some code
path to become more readable.
So far we have:
mod_rewrite
mod_redirect
mod_access
mod_auth
mod_cache
mod_chat
mod_status
mod_maps
28.11.2003 18:16
- redirects
as lighttpd supports url-rewriting redirection was a few lines of new code.
redirect ^/wishlist/(.+) http://jan.kneschke.de/wishlist/$1
rewrite ^/wishlist/(.+) /new/wishlist/$1
28.11.2003 17:00
- signal stuff
setitimer is used to send a event every second to call time() only once a
second. another system call which has been remove from the main-loop.
sending HUP to lighttpd will close and re-open the logfiles. this is used
for cycling logfiles.
#! /bin/sh
###
#
# a simple logfile rotator for lighttpd
#
DATE=`date +"%Y%m%d-%H%M"`
LPID=`pidof lighttpd`
mv access.log access.log.${DATE}
kill -HUP ${LPID}
gzip access.log.${DATE}
27.11.2003 01:07
- native win32 port
a first attempt for a native win32 has been done. For now mingw is the
base for the development as it provides a basic unix-like framework for
building native win32 applications.
the most internal files have been ported and the over all progress is
going well. At the end this will just be a prove of concept.
26.11.2003 01:17
- access denied
added a access-deny filter to block specific urls like
access-deny ~
access-deny .inc
17.11.2003 01:06
- bug fixing
a fstat() on a opened fd which has changed reports wrong the file-info.
Using stat() again helps to solve this problem.
the sig-io version doesn't suffer from this problem.
the etags are now used for verifing file-cache-entries.
- chat
enabled the internal login mechanism again
added support transfering session-infos over MySQL.
15.11.2003 00:19
- optimizing
added a buffer_equal_reverse function which is optimized version of strcmp
which is going backwards as pathnames are often the same for in the first
bytes.
wrote a one-pass parser for the request-header. The combination of
strstr(..., "\r\n") to seperate lines and strchr(..., ':') to seperate
keys from values more or less touched every byte twice.
we are still at 18.000 req/s for 4kb keep-alive requests even with etags
and handling all header fields.
14.11.2003 17:26
- fcgi
Content-Type wasn't passed correctly to the FastCGI app. HTTP_CONTENT_TYPE
was sent instead.
- cache
provided access to the Session-ID
- error-log
the timestamp is now written in a human readable form
22.10.2003 00:06
- fcgi, cgi
added a special set of array-functions which are optimised for the "insert
only once" case.
this provides access to the headers which are now forwarded to the external
interfaces. Before this change only a limited set of request-headers were
forwarded.
21.10.2003 11:58
- modules
added a new module-hook after the basic-init of the module for handling
config-settings and prepare the overall operartion (like building
db-connections, compiling regexes, ...)
- cache
use turckmm-cache 2.4.3 to get some numbers for the php-latency:
/usr/sbin/ab -n 10000 -c 10 http://alba.home.kneschke.de:1025/index.php
handling the cache-decision and the cache-hit in php:
cache-miss: 100% (-> $version = 0)
plain : 108.13 req/s
turckmm-cache: 218.39 req/s
cache-hit: 100% (-> $version = 1)
plain : 164.45 req/s
turckmm-cache: 653.98 req/s
handling the cache-decision and the cache-hit in the server:
cache-hit: 100%, but using index.cml
cml: 4918.84 req/s (no keep-alive)
cml: 6901.31 req/s (keep-alive)
cache-miss: 100%, but using index.cml
plain : 108.39 req/s
turckmm-cache: 217.84 req/s
Conclusion:
- there is no loss in the cache-miss case through the cml-handling
- the cache-hit case can be improved dramaticly with lighttpd-cache
- turckmm-cache improves the cache-miss case alot
20.10.2003 00:40
- cache
the first 'real-life' test showed dramatic improvements in the req/s
handling.
The basic idea was to move the decision if a php-page can be taken from
the cache from the php-code to the webserver.
See here why this is a good thing:
the quite common code which works for http://jan.kneschke.de/ is using
templates and is quite static, but depends on 4 external files (the
menu-structure, the template, the current content, the class-file).
the index-file is always:
include_once "jk.inc";
$v = new view();
print $v->get(array(array ("file" => "content.html")));
It is more or less the same for all pages.
This basic setup can deliver 100 requests/s.
The next step has:
- application bases caching
as we know that each pages depends on those 4 files, you can check if they
have been modified since the last request and deliver the content from the
cache otherwise.
this increased the throughput to 150 req/s. (cache-hit ratio 100%)
The next logic step is to move the decision-making process out of the PHP
code as PHP is to slow for the cache-hit path:
a CML (Cache-Markup-Language) has been written which describes the whole
decision process which has been written in PHP-code before:
output.content-type text/html
output.include _cache.html
trigger.handler index.php
trigger.if file.mtime("../lib/php/menu.csv") > file.mtime("_cache.html")
trigger.if file.mtime("templates/jk.tmpl") > file.mtime("_cache.html")
trigger.if file.mtime("content.html") > file.mtime("_cache.html")
if one of the 'trigger.if' statements is true the 'trigger.handler' is
called to generate the pages.
if none of the them is trigger the files from 'output.include' are sent to
the browser with content-type specified in the first line:
The result was very 'promissing':
5900 req/s with keep-alive
3800 req/s without keep-alive
(both for a cache-hit ratio of 100%)
for keep-alive this is factor <b>59</b> against the plain un-cached
version and still <b>39</b> againt the php-cache-version which is doing
exactly the same.
Time for party. :)
19.10.2003 - 18:55
- cache
the handling of functions has been improved. they are now 'plugable'. just
the dlopen() stuff is missing.
a new datatype has been added: the string
this makes it possible to evaluate something like:
trigger.if unix.time.now() - file.mtime("head.html") > 30
trigger.if mysql.query("SELECT count(*) " + \
" FROM structure AS struc, session AS sess " + \
" WHERE struct.user = sess.user" + \
" AND sess.id = \"" + mysql.escape("ab\"c") + "\"")
you see:
- string operations (concat)
- handing of escape-sequences
- functions
- comparisions
18.10.2003 - 13:39
- cvs
imported everything into the cvs server which makes the whole revision
handler a lot easier. The CVS server was up all the time but not used.
Importing required 5 minutes of work which included merge the freebsd and
the linux tree.
- compilers + platforms
on a regular various compilers and platform are check to compile with with
the current code base:
platform | os | compiler | state
---------+---------------------+--------------+---------
ia32 | Linux 2.4.22 | gcc 2.95.3 | ok
ia32 | FreeBSD 5.1-CURRENT | gcc 3.3.1 | ok
mips64 | IRIX 6.5 | gcc 3.2.2 | ok
misp64 | IRIX 6.5 | MIPSpro 7.41 | ok
the mipspro compiler revealed some warning which resulted in a nice
code-cleanup that made the code more readable.
18.10.2003 - 03:00
- e-tags and friends
in preparation for the php-conference at the begin of november in
frankfurt/main the server has the support some more caching/proxy tags
like:
- E-Tag (14.19) [done]
- If-Match (14.24)
- If-None-Match (14.26) [done]
- If-Range (14.27)
Section (13.3.3) binds them together. [RFC 2616]
using lxr.kde.org and lxr.mozilla.org revealed that
- konqui only uses if-none-match
- mozilla uses if-none-match and if-range
it looks like it isn't that easy to trigger the if-range case.
As ulf suggested the etag is a hash of file-size, inode-number and mtime.
- fd-caching
ulf just phoned me ask proposed the free the cache more agressivly in case
of fd-shortage. increasing the the fd-limit is the better idea. :)
17.10.2003 12:45
- chat
finished the mysql-support for storing the sessions
12.10.2003 20:56
- valgrind
used valgrind again to verify that the code a free of mem-leaks and found
a 'leak generator' in the chunk-api.
the last few leaks were just some missing free()'s at the end of the
program run which would have been freed anyway.
at the end valgrind couldn't find any missing free()'s.
11.10.2003 12:09
- FastCGI
reduced the number of system calls for FastCGI to WebServer to 2 calls per
fd-event. (ioctl() + read())
this has no direct effect of the performance of the server, but improves
the possible througput of the load-balancer.
10.10.2003 21:09
- FastCGI - load-balancing
a brown paper bug has been fixed which caused to decreasing throughput if
load-balancing was enabled.
benchmarking the req/s with load-balancing shows really nice results:
server : req/s comment
---------+--------------------------------------------------
ulf : 764.06 (php)
lappi : 800.06 (php)
ulf+lappi: 1526.95 (2 * php)
grisu : 1265.66 (php + ab + lighttpd)
all : 1647.72 (3 * php + ab + lighttpd)
all(nice): 1904.40 (same as all, but the local php on grisu
has been 'nice -20'd)
if a php is run on the load-balancer it has to get a lower priority than
the load-balancer itself as to handle the work of (here) 3 php-servers.
10.10.2003 15:11
- java ?
http://jakarta.apache.org/tomcat/tomcat-4.1-doc/jk2/common/AJPv13.html
08.10.2003 21:08
- gigE
Last week 3 RTL8169S gigE cards arrived and were installed in the
test-framework.
They are very cheap (20 Euros) and are a good start for a low-level
benchmark network.
First result show us:
48 Mb/s with ab.
The webserver and the test-server are equipped with a 2000+ AMD CPU. The
system load 95%, user load is 3%, the rest is idle on both plattforms.
After some calculations at gets clear that there are various bottlenecks:
1. The PCI-Bus (32bit/33Mhz) can only transfer 133Mb/s
- 48 Mb/s TCP-Traffic results in 55Mb/s Ethernet-Traffic (verified with
slurm) [outgoing]
- there is about 10Mb/s incomming traffic at the PCI bus which the
requests
- the rest of the devices at the PCI bus are eating the last few mb/s
2. IRQ-Handling
- There are reasons why the RTL8169S cards are so cheap
- they can't send jumbo-frames
- only 8k/64k buffers which results in a interrupt every 3 packets
- they are at least handling checksum offloading for ip, udp and tcp
In the end there is a need for new hardware to limit the throughput by the
Ethernet again.
- PCI-X, 64bit-PCI, ...
- 'real' gigE-Network cards
- ...
BTW: 2.4.22 + the r8139 driver are very flacky and resulted in 3 lock-ups
for today.
In another test the dual-PPro-FreeBSD 5.1-CURRENT machine has been used as
server with a gigE interface (re0) at has shown that the CPU is the limit
for this combination. The maximum thoughput was 7Mb/s.
Another small benchmark:
$ ab -n 1000 -c 10 http://192.168.2.41:<port>/lighttpd-20030925.tar
port | server | CPU Idle
------+----------------+-----------
1025 | lighttpd | 75%
1026 | thttpd 2.23b1 | 75%
1027 | boa-0.94.14rc17 | 69% *
1028 | apache 1.3.x | 77%
In all 4 cases the thoughput was 8600 kbytes/s.
* boa had 9 failed transfers.
- FreeBSD
The problems with FreeBSD 5.0-RELEASE vanished after updating to
5.1-CURRENT.
26.09.2003 18:22
- FreeBSD
A SMP-machine has been added to the test-farm. It is running FreeBSD
5.1-RELEASE and will help to the improve the scalability.
fixed sendfile() handling.
- FreeBSD problems
FreeBSD-5.1-RELEASE-SMP
2 * Pentium Pro 200 MHz
192.168.2.38 (doubleheart)
(webserver [lighttpd at port 1025, thttpd at port 1027])
Linux 2.4.20
1 * AMD 2000+
192.168.2.10 (grisu)
(ab)
Problem:
Connections are reset by the kernel without any application intervention.
Calling 'ab' (apachebench) at grisu with the following paramters:
/usr/sbin/ab -n 10000 -c 10 http://192.168.2.38:1025/index.html
results in the following output:
...
Time taken for tests: 40.610 seconds
Complete requests: 10000
Failed requests: 5980
(Connect: 0, Length: 5980, Exceptions: 0)
...
This is reproducable and the number of failed requests is always
5980 +/- 50 requests. In other words: after 4000 requests tcpdump shows
the following output:
tcpdump shows:
00:53:48.923029 192.168.2.10.39774 > 192.168.2.38.1025: S [tcp sum ok]
1013737315:1013737315(0) win 5840 <mss 1460,sackOK,timestamp 5208461
0,nop,wscale0> (DF) (ttl 64, id 7918, len 60)
0x0000 4500 003c 1eee 4000 4006 964d c0a8 020a E..<..@.@..M....
0x0010 c0a8 0226 9b5e 0401 3c6c 6763 0000 0000 ...&.^..<lgc....
0x0020 a002 16d0 eeaa 0000 0204 05b4 0402 080a ................
0x0030 004f 798d 0000 0000 0103 0300 .Oy.........
00:53:48.923330 192.168.2.38.1025 > 192.168.2.10.39774: S [tcp sum ok]
1803860672:1803860672(0) ack 1013737316 win 65535 <mss 1460,nop,wscale
1,nop,nop,timestamp 4459794 5208461> (DF) (ttl 64, id 6821, len 60)
0x0000 4500 003c 1aa5 4000 4006 9a96 c0a8 0226 E..<..@.@......&
0x0010 c0a8 020a 0401 9b5e 6b84 bac0 3c6c 6764 .......^k...<lgd
0x0020 a012 ffff d4ce 0000 0204 05b4 0103 0301 ................
0x0030 0101 080a 0044 0d12 004f 798d .....D...Oy.
00:53:48.924009 192.168.2.10.39774 > 192.168.2.38.1025: . [tcp sum ok] ack 1
win 5840 <nop,nop,timestamp 5208461 4459794> (DF)
(ttl 64, id 7919, len 52)
0x0000 4500 0034 1eef 4000 4006 9654 c0a8 020a E..4..@.@..T....
0x0010 c0a8 0226 9b5e 0401 3c6c 6764 6b84 bac1 ...&.^..<lgdk...
0x0020 8010 16d0 e9c3 0000 0101 080a 004f 798d .............Oy.
0x0030 0044 0d12
00:53:48.924150 192.168.2.10.39774 > 192.168.2.38.1025: P [tcp sum ok]
1:29(28) ack 1 win 5840 <nop,nop,timestamp 5208461 4459794> (DF)
(ttl 64, id 7920, len 80)
0x0000 4500 0050 1ef0 4000 4006 9637 c0a8 020a E..P..@.@..7....
0x0010 c0a8 0226 9b5e 0401 3c6c 6764 6b84 bac1 ...&.^..<lgdk...
0x0020 8018 16d0 27e4 0000 0101 080a 004f 798d ....'........Oy.
0x0030 0044 0d12 4745 5420 2f69 6e64 6578 2e68 .D..GET./index.h
0x0040 746d 6c20 4854 5450 2f31 2e30 0d0a 0d0a tml.HTTP/1.0....
00:53:48.924728 192.168.2.38.1025 > 192.168.2.10.39774: R [tcp sum ok]
1803860673:1803860673(0) win 0 (ttl 64, id 6831, len 40)
0x0000 4500 0028 1aaf 0000 4006 daa0 c0a8 0226 E..(....@......&
0x0010 c0a8 020a 0401 9b5e 6b84 bac1 0000 0000 .......^k.......
0x0020 5004 0000 64ba 0000
SYN, SYN+ACK, ACK, DATA, RST
strace shows that no connection attempt has been reported to the
application which is poll()'ing the server socket.
/* the common loop without any block attempts */
accept(3, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, [0]) = 5
fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
read(5, "GET /index.html HTTP/1.0\r\n\r\n", 4095) = 28
stat("/home/jan/lighttpd-0.1.0/servers/", {st_mode=S_IFDIR|0755, st_size=512, ...}) = 0
fstat(6, {st_mode=S_IFREG|0644, st_size=4348, ...}) = 0
write(5, "HTTP/1.0 200 OK\r\nConnection: clo"..., 235) = 235
write(2, "1064875136: (network.c.210) 235 "..., 33) = 33
syscall_393(0x6, 0x5, 0, 0, 0x10fc, 0, 0xbfbff2c0, 0) = 0
close(5) = 0
/* no futher waiting connections */
accept(3, 0xbfbff700, [1852702730]) = -1 EAGAIN (Resource temporarily unavailable)
/* enter the main-loop */
gettimeofday({1769235301, 1663069807}, NULL) = 0
poll([{fd=3, events=POLLIN}], 1, 1000) = 0
gettimeofday({4294967295, 65537}, NULL) = 0
poll([{fd=3, events=POLLIN}], 1, 1000) = 0
gettimeofday({4294967295, 65537}, NULL) = 0
poll([{fd=3, events=POLLIN}], 1, 1000) = 0
...
(strace is broken for accept() and gettimeofday(), syscall_393() is sendfile())
after entering the main-loop the webserver doesn't receive any new POLLIN
events at all for the next 10-15 seconds. Any connection-attempt within
the period is, as you can see in the tcpdump output, accepted by the
kernel and the received data is thrown away as the kernel sends a RST.
After those 10-15 seconds the application gets a POLLIN event for the
server socket and the normal data-transfer taken place for the next 4000
requests.
This behaviour is reproducable with thttpd 2.20c and the current lighttpd.
26.09.2003 11:42
- localizer
ported the localizer-server application into a module for lighttpd. After
30 minutes the code was ported and funtional. This modules provides access
to the localizer-db by a HTTP-Interface and generates the response as
HTML, CSV and plain-text.
So far, the module-interface looks good and flexible enough.
7000 req/s is good enough too. Generating the HTML and querying the DB
needs some time.
22.09.2003 08:40
- modules
introduced a simple module interface which allow to hook into the process
of handling the requests. Basicly it allows to move the url-rewriter, the
auth-sub-system and the cache out of the main-code. The module-interface
will be extended to allow the cgi and the fastcgi sub-processes to be
moved into a module.
15.09.2003 09:36
- error-handling
fastcgi and cgi connections are now closed correctly if the corresponding
client-connection has died.
14.09.2003 10:40
- cgi
finally streaming works with CGI, too
this enable support for cgiirc.sf.net and friends which use streaming as
there transport mechanism.
streaming has been verified with cgiirc-0.5.2.
- fdevents
there where some reports that sigio didn't work as expected. It just
reported no events at all. Looks like a known bug in the glibc on
those platforms.
Using poll() instead solve the problems.
27.08.2003 22:12
- rewrite
a pcre based rewrite engine has been integrated
- cgi
the file-based cgi interface has been replace by two pipes.
23.07.2003 13:29
- fdevents
the whole fd-event handling has been reworked and several smaller bugs
and design-errors have fixed.
sigio, poll() and select() are working fine again.
On IRIX 6.5 SIGIO could be used, but without further testing poll() is used.
- cgi
CGI-support is still broken.
21.07.2003 18:46
- dnotify
using the F_NOTIFY feautre of the Linux 2.4.x kernel gives anothre nice
performance boost as lighttpd can cache the stat()'s the right way, now.
23009.66 @ 5-4k-k which means +10% against 30.06.2003 14:03
for 1000-4k-nok the performace nearly doubles: 3730.23 goes up to 6144.39
17.07.2003 13:21
- FreeBSD
a small patch (#include <>) to compile lighttpd on FreeBSD again.
08.07.2003 10:48
- fileinfo-cache
A reallife test showed that the cache wasn't perfect at all as it made
lighttpd crash. This is fixed now.
30.06.2003 17:40
- lighttpd-bench
After some problems with other benchmarking tools for webservers the first
version of lighttpd-bench has been written.
It a revealed a nasty strange behaviour which was fixed by increasing the
listen-backlog from 5 to 1024.
30.06.2003 14:03
- fileinfo-cache
the fileinfo-cache has been relaxed a little bit and there are always 2
fstat()'s per file-request now. This isn't that dramatic:
21800.74 req/s is still a very good result (7-4k-k).
29.06.2003 03:29
- fileinfo-cache
a fileinfo cache has been added to reduce the number of system-calls to
stat and open a file.
in the releases before the same file was stat'ed at least 2 times plus a
stat on the docroot for each request. Now the stat()'s and the
corresponding open() + close() calls are cached and the number of system
calls has been reduced to the minimum:
$ strace -eopen,stat64,read,write,sendfile,accept,shutdown,close \
-p `pidof lighttpd`
/* first connection */
accept(3, {sin_family=AF_INET6, sin6_port=htons(56211),
inet_pton(AF_INET6, "::ffff:192.168.2.10", &sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, [28]) = 5
accept(3, 0xbffff470, [28]) = -1 EAGAIN
read(5, "GET /index.html HTTP/1.0\r\nUser-A"..., 4095) = 91
stat64("/home/weigon/projects/lighttpd/servers/grisu.home.kneschke.de:1025/pages/",
{st_mode=S_IFDIR|0755, st_size=3656, ...}) = 0
stat64("/home/weigon/projects/lighttpd/servers/grisu.home.kneschke.de:1025/pages//index.html",
{st_mode=S_IFREG|0644, st_size=4348, ...}) = 0
open("/home/weigon/projects/lighttpd/servers/grisu.home.kneschke.de:1025/pages//index.html",
O_RDONLY) = 6
write(5, "HTTP/1.0 200 OK\r\nConnection: clo"..., 235) = 235
sendfile(5, 6, [0], 4348) = 4348
shutdown(5, 1 /* send */) = 0
close(5) = 0
/* second connection */
accept(3, {sin_family=AF_INET6, sin6_port=htons(56212), inet_pton(AF_INET6,
"::ffff:192.168.2.10", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28])
= 5
accept(3, 0xbffff470, [28]) = -1 EAGAIN
read(5, "GET /index.html HTTP/1.0\r\nUser-A"..., 4095) = 91
write(5, "HTTP/1.0 200 OK\r\nConnection: clo"..., 235) = 235
sendfile(5, 6, [0], 4348) = 4348
shutdown(5, 1 /* send */) = 0
close(5) = 0
In the end we have a new speed record:
(ab -c 8 -n 100000 http://192.168.2.10:1025/index.html with poll())
Requests per second: 23435.67 [#/sec] (mean)
This is a speed increasement of 20-30% against the last internal benchmark.
28.06.2003 02:30
- caching framework
the trigger.if directive is working fine. the basic algebra is done ( +,
-, *, / and comperations =, <=, >=, >, < and the boolean logic
&& and || including braces)
two basic function are available:
- unix.time.now
- file.mtime(...)
25.06.2003 17:33
- caching framework
added output.include, output.content-type for a cache-hit and
trigger.handler for a cache-miss.
the actual decision is made be trigger.if which will be added tomorrow.
15.06.2003 15:33
- sig-io
After several other smaller optimisations lighttpd performs better with
sig-io under high load:
sigio+poll poll
c1000-4k-nok : 2635.95 1643.39
c1000-4k-k : 7335.68 6788.87
c1000-100k-nok: 2353.49 1217.73
c1000-100k-k : 3097.89 2080.73
The user-space part has been optimized a lot. Now it is time to optimized
the number of context-switches between user and kernel-mode.
user 0m0.180s
sys 0m1.160s
a fileinfo-cache is the way to go.
struct {
buffer *name;
struct stat st;
int handler;
...
int fd;
void *mmap_p;
}
11.06.2003 14:57
- sig-io is back again
and it works fine.
sigio (Realtime-Signals under Linux 2.4.x) sends one signal per event and
buffers the rest of the events in a kernel queue. If the is full a SIGIO
is sent and poll() gets all events at once for further processing.
Currently the behaviour is a little bit strange:
- sig-io + poll() is good for non-keep-alive connections
- poll() for keep-alive connections
c1000-4k-nok: (1000 concurrent request, 4k filesize, no keepalive)
poll : 1521.38
sigio+poll: 2124.00
c1000-4k-k: (1000 concurrent request, 4k filesize, keepalive)
poll : 5882.35
sigio+poll: 1239.46
Very strange for now.
09.06.2003 23:59
- code-cleanup
the event-handling code has been rewritten to handle single events better
as they are expected from sig-io.
the fallback-mode of sig-io is broken now, but the normal poll() mode got
a 10% increasement in speed. This means the we are back the speed level of
20030308-0155 are as fast as zeus again.
Especially under higher load the current lighttpd performes better.
09.06.2003 11:51
- sig-io benchmark
20030609-1151 20030608-2110 20030308-0155
-c 10 sigio select() poll() poll()
4k : 7870.92 7937.77 8035.36 9443.76
4k (keep-alive) : 14098.41 14590.02 14275.52 17985.61
100k : 3366.32 3382.03 3261.15 3722.32
100k (keep-alive) : 5544.77 5576.00 5573.20 5975.86
-c 100
4k : 6144.77 5821.40 5714.29 6724.95
4k (keep-alive) : 9097.53 9213.19 8979.08 10833.06
100k : 2549.33 2495.94 2318.95 2607.36
100k (keep-alive) : 4267.67 4283.94 4094.17 4314.06
For -c 100 4k and 100k sig-io gives a small increasment.
09.06.2003 01:00
- sig-io
writing large files works now as expected. After removing the limit for
the chunks of sendfile the write-buffer-is-empty-again signal is
generated. that was missing.
321 of 10000 connections still have the wrong length. After that is fixed
it is time to start some benchmarks again.
08.06.2003 21:10
- sig-io
first attempts in getting SIGIO support running which were not very
successfull yet.
- poll()/select() benchmark
20030608-2110 20030525-1623 20030308-0155
-c 10 select() poll() poll() poll()
4k : 7937.77 8035.36 8166.60 9443.76
4k (keep-alive) : 14590.02 14275.52 14781.97 17985.61
100k : 3382.03 3261.15 3176.42 3722.32
100k (keep-alive) : 5576.00 5573.20 5809.56 5975.86
-c 100
4k : 5821.40 5714.29 5669.26 6724.95
4k (keep-alive) : 9213.19 8979.08 8418.22 10833.06
100k : 2495.94 2318.95 2314.28 2607.36
100k (keep-alive) : 4283.94 4094.17 4456.92 4314.06
as the overall processing cycle has been rearranged the overall figures
changed in minor ranges. c100-4k-k increased, *-100k-k decreased.
At least it still works.
08.06.2003 12:45
- select()/poll()
implementate an abstration layer for fd-events (like eevry other webserver).
currently we support poll() and select().
This should bring us support for Mac OS X and propably Windows.
04.06.2003 18:35
- configure
lighttpd can now be build without ssl-support even if openssl is installed
- protability
- on IRIX IPv6 is supported now out of the box
- for FreeBSD some missing haeders have been added
04.06.2003 14:34
- pipelining
adding support for pipeling introduces a problem if the request-header
was broken. this has been fixed now.
- unneccesary slashes
when building pathes from different portions of a string (e.g. docroot +
virt-server-docroot + request-uri) slashes where added for security and
simplicity. This resulted in up to 5 adjacent slashes which caused no harm
but looked strange. (fixed)
04.06.2003 09:57
- start/stop messages
the error-log contains the start and end-times of the lighttpd process,
now.
- configfile
the config-file parser has been relaxed to accepts tabs instead of spaces.
- better error-handling
fixed a <a
href="http://bugs.php.net/?id=24009">bug in the FastCGI-SAPI of PHP</a>
if the fastcgi process dies or closes the connection unexpectedly we
return 500 now instead of closing the connection to the client.
- Location
the CGI/1.1-rev-03 specification requires us to send Status 302 if a
Location-header is sent by the client and a Status-header is missing.
- PATH_TRANSLATED
if PHP is compiled without --discard-path PATH_TRANSLATED has to be
provided.
27.05.2003 15:54
- directory listings
if a directory is requested and the directory doesn't contain a index-file
a directory-listing can be displayed. You have to enable directory-listings
in the config-file (directory-listings on)
- url-decoding
up to know urls where not decoded at all (%26 -> . and so on). This has
been added. Unicode isn't supported as we use 8-bit chars internally.
26.05.2003 00:44
- pipelining
as Sascha required pipelining for his benchmarking tool it has been
implemented.
pipelining allows sending a bunch of requests at once without waiting for
the actual responses. This reduces the network-overhead and the
round-trip-time.
- non-free()d memory
dmalloc helped to close some non-free()d memory. For the normal operation
this isn't important as only memory chunks which had to be free()d at the
end of the live-time of the lighttpd-process were not de-allocated.
- partly-initialized variables
lighttpd wasn't initializing the main-structure which resulted in strange
behaviour in rare circumstances.
25.05.2003 16:23
- benchmarks
after removing some useless internal copies we are more or less at the old
speed levels.
after adding virtual-hosts 10% of the performance were lost. Using less
memcpy() operations might add several other boosts.
I've just checked how lighttpd compares to Zeus.
lighttpd (current) (old)
-c 10 20030525-1623 20030308-0155 Zeus 4_2
4k : 8166.60 9443.76 7278.55
4k (keep-alive) : 14781.97 17985.61 16496.21
100k : 3176.42 3722.32 3156.37
100k (keep-alive) : 5809.56 5975.86 5460.30
-c 100
4k : 5669.26 6724.95 5134.26
4k (keep-alive) : 8418.22 10833.06 8010.25
100k : 2314.28 2607.36 2688.32
100k (keep-alive) : 4456.92 4314.06 4240.70
23.05.2003 14:38
- cgi-variables
HTTP_HOST was missing for the cgi-module while the fcgi-module passed it
through to the handler. Fixed that.
- fcgi-errors
the connection to the fcgi was dropped and poll() reported an error, the
error wasn't reported to the client the right way.
22.05.2003 23:02
- authorization
the first password-storage has been added:
[auth]
backend plan
plain-userfile <filename>
require /download/ user=jan|user=anom
http-auth /download/ "download archiv" digest
groups are prepared but not implemented. basic and digest are working fine.
20.05.2003 17:53
- authentification
The auth-methods from RFC 2617 have been added.
- auth basic
- auth digest
The only source for accounts is currently only the config-file.
auth-digest needs the plain-text passwort. Are there any source which
provide a plain-text password ?
12.05.2003 14:33
- virtual hosts
added very basic virtual-host support
virtual-server-root /home/weigon/projects/lighttpd/servers/
virtual-server-default-host grisu.home.kneschke.de:1025
virtual-server-docroot /pages/
docroot is
- if http-host exists
<virtual-server-root> + <http-host> + <virtual-server-docroot>
- otherwise
<virtual-server-root> + <virtual-server-default-host> +
<virtual-server-docroot>
- if even virtual-server-default-host does not exist, 500 is sent
12.05.2003 13:02
- code cleanup
After two month of development it was time clean-up the internal
structures. It looks like every went fine as lighttpd works es expected
like before.
- deflate
the on-the-fly compression has been verified to work fine with opera,
konqui, mozilla and the IE.
12.05.2003 02:10
- on-the-fly compression: deflate
Why the hell are the defining a "deflate" encoding in the form of
_deflate_ The "zlib" format defined in RFC 1950 [31] in combination
with the "deflate" compression mechanism described in RFC 1951 [29].
(RFC 2616)
and noone implements it that way ? Konqui and Mozilla expect a plain
deflate() package without the zlib-header.
Konqui is using "inflate2(..., -MAX_WBITS); " which is noted in the zlib
source as
/* handle undocumented nowrap option (no zlib header or check) */
Funny. Very, very funny.
Anyway. We have mimetype-depended compression support now.
11.05.2003 21:56
- logging
re-arranged the logfile structure to write CLF + useragent + referrer.
11.05.2003 10:23
- POST file-upload
added the missing functionality to send more than a single packet to the
FCGI-Server (or Client ? ... the PHP).
This gives us the file-upload thing for eg. PHP and large user-forms
(>16kb).
05.05.2003 15:21
- PATH_INFO
added support for PATH_INFO. PHP is a little strange and doesn't trust the
passed PATH_INFO setting. Works fine now.
30.04.2003 15:25
- bug-fixing day
While testing the FastCGI interface with the MSIE Björn Schotte discovered
that sometimes the output repeats itself from the start in an endless loop.
This bug has been fixed.
The read-write-fallback for ancient systems seeked the wrong FDs.
The FastCGI handler is now separting the HTTP-Header from the
response-body what results in a cleaner interface. The "header too long
for caching" message is gone now.
28.04.2003 18:18
- chunked transfer-encoding
The FastCGI part is now using Chunked-Transfer-Encoding if HTTP/1.1 is
used and no content-length is specified.
27.04.2003 23:10
- chunked transfer-encoding
added "Transfer-Encoding: chunked" which is currently used in the web-chat
for the endless stream. Perhaps it helps some browsers.
The FastCGI interface will get a the chunked-support too, as it will
enable keep-alive even if no content-length is returned from the FastCGI
process. We know the size of the chunks and will report it to the browser.
16.04.2003 12:02
- gigE deatchmatch
sascha compared the current lighttpd to his premium thttpd in his
gigabit-Ethernet-Network.
<a
href="http://schumann.cx/gbit_deathmatch.txt">http://schumann.cx/gbit_deathmatch.txt</a>
small comment:
lighttpd provides the same performance (req/s and thoughput), but uses more
CPU-Time.
10.04.2003 17:22
- works on IRIX
Jörg Behrens provided me login to his SGI Origin and after fixing two small
typos it worked fine on IRIX. (#if define HAVE... was missing a 'd' and
getopt returns a 'int' and not a 'char')
IPv6 support is currently disabled for IRIX a gethostbyname2() isn't
available.
10.04.2003 15:56
- another bug-fixing day
The upper limit of open connections was enforced and a caused a seg-fault.
The current limit is set to 4096 parallel connections.
As sascha benchmarked lighttpd and his premium thttpd I tried to reproduce
his reported results and had to fix 2 flaws in the async-io handling of
httpd_load which was used for the testing.
If sascha starts another benchmark session I'll put a link to it here.
09.04.2003 00:08
- bug-fixing time
Date: and Last-Modified: where sending a timezone != GMT what was invalid.
The timestamp itself was correct, just the timezone use the wrong
characters.
the fcgi-code had an buffer-overflow for larger POST-Requests.
The fcgi-code still has problems with POST-Request larger than 16kb as the
the write buffer is full. Currently we don't handle this case except from
reporting it in the error-log.
02.04.2003 01:17
- cgi is back
The CGI Interface wasn't tested for a longer time. It don't really
survived the introduction of the config-file handling and the internal
changes that were part of it.
A small test with
#! /usr/bin/perl
print "Content-Type: text/html\r\n\r\n";
print time()."\n";
0;
produced 219 req/s.
Calling PHP via the CGI interface results in 100 req/s. Same script, same
parameters for 'ab' and same PHP result in 1400 req/s if we use the
FastCGI interface.
<b>Don't forget</b>: these benchmarks only represent figures for scenarios which
are not very realistic. They are just usefull for comparisions of the
internals. We don't want to benchmark applications.
01.04.2003 23:04
- new benchmarks
This time we wanted to see if we can get the fastcgi-Interface a little bit
faster. We use this small script for the testing the interface.
<?php
ob_start(/*"ob_gzhandler"*/);
print "12345<br />\n";
header("Content-Length: ".ob_get_length());
ob_end_flush();
?>
It generate a small network load and is the best case for benchmarking the
overhead of the call to the fastcgi-php.
Using the chunk-API instead auf tmp-files increase the req/s vom 1200req/s
to 1600 req/s and the IO-load went down as expected. Waiting for the
filesystem and creating 1200 files per second required some time. The
CGI-Interface is still using tempfiles.
01.04.2003 19:28 (no april fools joke)
- added SSL support
I took -lssl and -lcrypto and added SSLv2/SSLv3/TLS support to lighttpd.
After some reading I realisized that libcrypto contained some of my code.
They have a similar buffer-struct with similar functions and they use the
same MD5-code from the RFC. :)
Adding basic SSL-support was quite esay: replacing all write/read-calls by
SSL_write/read, initializing the ssl-context the right way and telling
openssl where to get the data from (SSL_set_fd). It fits really well into
lighttpd.
- sidenote
sooner or later the con->filename handler will vanish in favour of the
direct interface to the chunk-API which will simplifiy the design a little
bit.
31.03.2003 20:50
- added a web-chat module
1400 lines of C-code are neccesary to add a web-chat to lighttpd.
Features:
- bb-code for text-layouting
- auto-highlight for URLs and Email-adresses
- unlimited number of channels
- max. 32.000 users per channel
- fast as it is directly integrated into the web-server
- easy to install (as easy as lighttpd)
- IRC-like (/msg, /kick, /me, ...)
- works with MS IE, Mozilla, Konqueror, ...
- uses as little JavaScript as possible to stay compatible with all browsers
- flood-protection (2 Levels: ignore and kick)
- uses CSS for customized look-and-feel
25.03.2003 00:36
- finished the config-file support
now we have config-sections
[fastcgi]
fastcgi .php 192.168.2.76 1025
The only this missing is -HUP handling to re-read the config. But that
will follow when I'm up again.
20.03.2003 01:04
- adding config-file support
the most boring part of the writing programs after writing documentation.
The format is quite simple and good enough for lighttpd.
19.03.2003 03:05
- more on sendfile support
added support for freebsd's version of sendfile().
Linux:
ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);
FreeBSD:
int sendfile(int fd, int s, off_t offset, size_t nbytes,
struct sf_hdtr *hdtr, off_t *sbytes, int flags);
the first 2 params are swap, the next 2 are the same and the last 3 are
set to NULL or 0.
Solaris 9 is providing a direct interface to the chunk-API with the
sendfilev() system call:
ssize_t sendfilev(int fildes,
const struct sendfilevec *vec, int sfvcnt, size_t *xferred);
recovering from a failure is a little bit more complex, but it should be
fast. I just need a system to test it. Anyone ?
18.03.2003 17:32
- in the news
what a surprise: lighttpd is announced in entwickler.com.
<a href="http://www.entwickler.com/itr/news/psecom,id,9483,nodeid,82.html">http://www.entwickler.com/itr/news/psecom,id,9483,nodeid,82.html</a>
this was not planned.
- implemented the chunked-API
rather writing the content of multi-range'd requests to a temp-file we
just register the parts in a chunk-queue.
chunk 1 [mem] - part-header
chunk 2 [file] - source-file (offset, len)
chunk 3 [mem] - part-header
chunk 4 [file] - source-file (offset, len)
chunk 5 [mem] - part-footer
If everything is prepared, the content-length is calculated and the
http-header is generated and prepended to the chunk-queue.
the handle_write() function which handles the write-process just sends the
chunks to the network without only further modifications.
the chunk-api even simplifies the
'my-fastcgi-process-needs-some-time-to-create-the-response-
and-sends-the-data-in-chunks' case. Just add the chunk to the queue if it
is received.
13.03.2003 10:49
- another set of real-life benchmarks
the chairman was benchmarked a little bit. Chairman is full-flegded WCMS.
As every CMS it needs some time to generate pages. To speed the page
generating the static-parts of the page can be cached internally.
without caching with caching
apache + mod_php : 2.50 10.64 req/s
lighttpd + fcgi-php: 2.50 22.74 req/s
Looks like lighttpd is the way to go :)
Please, don't compare these numbers with the other benchmarks:
We used a different test-machine which is a little bit under-powered
for this test-case.
12.03.2003 13:43
- another large application works with lighttpd
<a
href="http://www.thinkphp.de/content/content2.php?CatID=44&NewsID=95">Chairman</a> from <a href="http://www.thinkphp.de/">thinkphp</a> has been verified to work with lighttpd.
Three things are worth to note:
1. ALWAYS use log_error in the php.ini if you use FastCGI
otherwise it will put the errormessages directly into fastcgi-socket
without any FastCGI Header. This will result in strange errors.
(its a PHP bug)
2. DOCUMENT_ROOT has been added to the environment variables
3. a small bug for sending larger output from a fastcgi client has been
fixed.
11.03.2003 11:52
- added two neccesary features
- redirect _dir_ to _dir_/
- append index-file to _dir_/ if they exist
11.03.2003 00:13
- another target
after adding some headers lighttpd compiles without any warnings
under cygwin on Windows.
A native windows port will take some time as I have to figure out the
changes on the winsock.h to the unix headers.
10.03.2003 11:52
- added Solaris 8 and 9
lighttpd has been confirmed to compile and work on
- linux
- FreeBSD, NetBSD
- Solaris 8, 9
Perhaps I can get lcc to compile it on windows.
10.03.2003 01:30
- another platform was confirmed
lighttpd works on NetBSD
- another webserver benchmark
mathopd (http://www.mathopd.org/)
Mathopd/1.4
-c 10
4k : 6329.11 [#/sec]
4k (keep-alive) : 10235.41 [#/sec]
100k : 1168.50 [#/sec]
100k (keep-alive) : 268.82 [#/sec] (99% idle)
-c 100
4k : ---- (connections dropped)
4k (keep-alive) : ----
100k : ----
100k (keep-alive) : ----
Very good values for small files, but the rest ?
10.03.2003 00:06
- added more write-handlers
1. sendfile (linux only)
2. mmap + write
3. read + write
This means that lighttpd runs on some more systems. It has been verified
to compile and work on linux 2.4.x and FreeBSD.
-c 100
sendfile mmap read
4k 6476.68 5698.01 5363.08 [#/sec]
100k 2312.35 841.54 783.09 [#/sec]
The mmap() numbers could be better with a fd+mmap-cache. The zero-copy thing
is good for our performance.
- tested another small httpd - BOA
just to get a better view about the performance of lighttpd i've tested
another httpd: Boa/0.94.14rc16 (http://www.boa.org/)
Boa/0.94.14rc16
-c 10
4k : 5659.63 [#/sec]
4k (keep-alive) : 250.23 [#/sec]
100k : 1104.29 [#/sec]
100k (keep-alive) : 1363.14 [#/sec]
-c 100
4k : 4319.65 [#/sec]
4k (keep-alive) : 2490.66 [#/sec]
100k : 815.93 [#/sec]
100k (keep-alive) : 1007.05 [#/sec]
The results are a little bit strange. Keep-Alive for small files seems to be
broken.
It looks like lighttpd has enough power for now. Time the port and add more
features.
09.03.2003 13:26
- load-balancing works great
I used the framework from http://framework.netuse.de/ as a
real-life example again to test the load-balancer.
setup:
- grisu
- AMD athlon XP 2000+
- runs lighttpd + ab + a 'nice -15'ed FastCGI-PHP
- generates 153 req/s alone
- laptop
- Intel PIII 850
- runs a FastCGI-PHP
- generates 88 req/s
using the internal load-balancer of lighttpd which balances the
php-requests over grisu and laptop generate:
221 req/s
An apache with mod_php running on grisu gives 117.04 req/s only.
09.03.2003 12:46
- building the connect connection to the fcgi-php is not non-blocking
socket() -> connect() -> fcntl(non-blocking) resulted in some problems as
the connect() call blocked sometimes for 1 seconds.
socket() -> fcntl(non-blocking) -> connect() solves this problem, but
addes more overhead. The first connect-attempt seems always to
return EINPROGRESS.
08.03.2003 15:06
- the first 'real-life' PHP test showed 2 bugs in the POST handling.
1. the Content-Type header was not forwarded via fastcgi
2. the internal content_length handler wasn't reset after the request
now lighttpd + php work with
http://framework.netuse.de/
the basic application gives us 145 req/s
08.03.2003 11:06
- use diet to 'test' your code
diet gives some use full warnings which should be followed. I took the
chance and removed the last few fprintf() and sprintf() from the code.
now, we don't need stdio.h anymore.
a staticly linked, stripped lighttpd is 42884 bytes large.
08.03.2003 01:55
- buffer_strcat() uses strlen() to get the length of the 'to-be-catted-string'
if the string is constant or has a known length you can pass that function
which gives use buffer_strcat_len() which passes the string length as a
parameter.
- using a ramdisk as the base for the tempfiles generated mkstemp() is a
very cood idea
it reduces the io-load on the system and gives as more cpu-time for
load-balancing the php-requests.
- cache the output of strftime(), gmtime() and localtime
we generate more then 10.000 req/s which results in using the same
timestamp over and over again. It is sufficient to generate the timestamp
once a second and give us a bunch of CPU-cycles for sending files.
the same applies for other timestamps like Last-modified which sends a
string version of st.st_mtime. Cache it.
- don't try to overoptimize for code for the sake of clean code
the range support require some hack in the first versions like setting the
write_offset to the start of the range and keeping the rest of the
"code 200" send code as usual. Only a few lines added this first range
feature. fast, but a hack. It made things more complicated then neccesary
to add multi-range support in this scenario.
removing the hack doing small clean preprocessing helps to clean up the
whole 'write a chunk on the wire' code. At the end we use less code and
have a cleaner design.
Sascha proposed a chunked layer which just takes the chunks (http-header,
part-header, part-message, ... and the last boundary) and passes them to
the write-to-the-wire-code. This is far more elegant and will perhaps same
us from other problems, too. Currently I think that it is not necessary.
First we have to fix some protability issues.
Ok, today benches:
today last
-c 10
4k : 9443.76 7739.94 + 22%
4k (keep-alive) : 17985.61 13885.03 + 29%
100k : 3722.32 3349.97 + 11%
100k (keep-alive) : 5975.86 4965.49 + 20%
-c 100
4k : 6724.95 5918.56 + 13%
4k (keep-alive) : 10833.06 8405.48 + 28%
100k : 2607.36 2393.60 + 9%
100k (keep-alive) : 4314.06 4035.35 + 7%
Top Transfer Rate:
86365.77 [Kbytes/sec] received
Too much for a GigE-link ?
so far:
- use state-engines
a non-forking webserver like described in
http://www.kegel.com/c10k.html
- don't use fprintf() for logging.
writing one line to log file takes more time that sending responsing the a
HTTP-request
- take care of your memory
typedef struct {
char *ptr;
size_t used;
size_t size;
} buffer;
buffer* buffer_init();
void buffer_free(buffer *b);
int buffer_prepare_copy(buffer *b, size_t size);
int buffer_prepare_append(buffer *b, size_t size);
int buffer_strcpy(buffer *b, const char *s);
int buffer_strcpy_len(buffer *b, const char *s, int s_len);
int buffer_strcat(buffer *b, const char *s);
int buffer_strcat_len(buffer *b, const char *s, int s_len);
buffer_init() is only called once per buffer. If you don't need the
buffer, set 'used' to zero and reused it afterward.
buffer_strcpy() + buffer_strcat() check if the 'used' + the new strlen()
fit in to the 'size' of the buffer. If not the realloc() the buffer.
buffer_strcat() uses memcpy(ptr + used - 1, s, s_len + 1); which should
be faster than the original strcat().
Using those buffers keep the memory usage at 2Mb. malloc() and free() are
only called at the beginnig of the programm and at the end.
- don't used sprintf() to convert a integer into a string as like
sprintf(buf, "%d", l);
write your own ltostr function. sprintf() is to general for the case and
is slow.
Speed:
------
Let's assume that <a
href="http://www.acme.com/software/thttpd/">thttpd</a> is (one of) the
fastest webservers out there.
all httpds were started with:
$ ./thttpd -p 1026 -D \
-d /home/weigon/projects/localizer/src/lighttpd/docroot/ \
-l thttpd.access.log
(lighttpd uses the same parameters).
We used ApacheBench for testing the RPS (requests per second).
$ /usr/sbin/ab -dS -c 10 -n 100000 http://192.168.2.10:80/dummy.out
lighttpd thttpd/2.21b+php thttpd/2.20c thttpd/2.23b1
(concurrency 10)
4k : 7739.94 6040.84 5078.20 5888.24 [#/sec]
4k (keep-alive): 13885.03 10349.82 5034.49 5853.09 [#/sec]
100k : 3349.97 1176.29[1] 1188.74 1198.29 [#/sec]
100k (keep-a.) : 4965.49 2513.38[2] 1100.65 1130.51 [#/sec]
(concurrency 100)
4k : 5918.56 4907.01 4987.53 4886.87 [#/sec]
4k (keep-alive): 8405.48 6379.99 4938.76 4816.26 [#/sec]
100k : 2393.60 972.73 958.27 961.09 [#/sec]
100k (keep-a.) : 4035.35 893.51[3] 970.21 955.05 [#/sec]
thttpd/2.21b+php has been patched with the keep-alive + php patches from PHP4
sapi/thttpd/thttpd_patch.
[1] ab reported:
Failed requests: 1
(Connect: 0, Length: 1, Exceptions: 0)
[2] ab reported:
Failed requests: 5
(Connect: 0, Length: 5, Exceptions: 0)
[3] ab reported:
Failed requests: 24
(Connect: 0, Length: 24, Exceptions: 0)
Tunning the thttpd:
-------------------
- disable symlink checking (chroot() or -nos)
- log to /dev/shm/logfile, a ramdisk or disable logging
$ ./thttpd -p 1026 -D \
-d /home/weigon/projects/localizer/src/lighttpd/docroot/ \
-l /dev/null
