Goodbye slowness! The box had crept to a standstill, for some unknown reason. I thought it might be a DDOS or php script memory problems, but after those were avoided, the problems persisted. I finally had enough and decided to fix things for good. I've been using a Intel X25-m on my workstation for over a year, and it has done a great job, so now the server has one. More than that, however, I built a vmware-server based host on ubuntu, which was made much easier with this great patchset. VMWare server is an awesome unsung gem, once it's configured and working properly. It runs VMs, allowing direct VM access to the outside world (via bridged networking) and full instance management. So now, there's three new OS install running on the server... one for BeNOW, one for CJSW and one for other vhosts. The segregation provides security and isolation and, as they are running off the SSD, they are as fast, if not faster than tradition hardware bound OS's. Backup and expansion is also made easier. It's a funky setup, with vmdks on raid and selective direct disk access. The migration into the different VMs was quite complicated, as there is much customization that has occured over the years. It's now one step closer to an enterprise setup... overkill, perhaps, but it's working well, and should be great for years.
The box is currently being burdened by bad php on vhosts. Congratulations for waiting to see this. Things should improve shortly.
I played with adding interactive profiler support (via JIP) this weekend. The eventual goal is to be able to selective enable profiling of web applications as needed, so as to find slow segments of code, slow running tasks, etc. JIP works well and can quietly sit and wait for invocation. When started, timing information is gathered (this method took X ms and consisted of calls to these methods which took Y,Z ms) until profiling is stopped. There is some slowdown (2x perhaps) while profiling but not much slowdown at all when not profiling... which means it can run and wait for profiling.
The main overhead of profiling is figuring out useful information from the reams of dumped data. The data is there, but finding relevant code sections (needles) to analyze for optimization amid the huge dump (haystack) can be tricky. The best way I could think of is to provide some sort of visualization frontend... a way to quickly identify problem segments. I think I've found a solution with inline SVG. SVG in combination with scripting, ajax and data fetch could be quite useful. Here's a demo of what I've managed so far (requires recent firefox):
It works very well and when dynamically driven should be an excellent way to whittle down the data to the important. I'm going to try to whip up something similar to the disk usage chart found in boabab.
Building on the newly added extension mechanism, I've added early auto-translation support to the web framework. It already performs basic translation using google translate api. As shown here, it automatically translates easily specified blocks of text:
It's very basic now, but will have more features added shortly. Currrently, the Accept-Language header is read and a block of text is translated to the requested language. Soon, the translated text will be stored for further translations. Eventually, translation maintenace should be supported (ie translator users providing better translation) as well as facilitated tools for page developers to easily make existing pages translable.
I've also ported the existing svc tag to web:svc and the extension mechanism is proving very fast and easy to work with.
I've created an easy XSL extension mechanism and added it to the web framework. It allows for easy coding of things which output stuff to pages during the rendering processes. As the extensions are java based, they may do complex processing and return the result. Custom extensions should be quite useful for advanced derivation of content. The primary reason driving the development of this extension mechanism is to provide dynamic translation. I'm hoping to soon write up the code to translate text within the dynamic pages. Translation will be fixed but will also use google translate to translate text on demand for newly requested languages. Ideally, automatic translation will be refinable by translator users who can smooth things out. What it should mean is that the application (and modules that make up the application) are automatically available in multiple languages with the best translation possible. The web framework extension mechanism should prove useful for many things and while it builds on the xalan extension mechanism it is much easier to use and maintain. Documentation is currently lacking, but will be included in the tutorials eventually. It's another obscure feather in the cap that is the web framework.
It was a great, if moist party. Many regulars I know did not attend this year. A few were there (Boucher, Elaine) and I made a few newfriends. There was rain each night, but nice days on Saturday and Sunday. As usual, the people were loving and friendly, the techno was prime and the sound systems were pumping. The recording went off nearly faultlessly. I recorded Elaine's set at noon(ish) on Friday, then the main stage for the rest of the weekend. I turned it on and let it go on Friday, which left much silence on Saturday morning. I was recording 16/44.1 WAV and it split it up into 2GB chunks. On Saturday night I used the line trigger, but it was set a little high, so there were many small files, but they seem to be close enough to be put back together. I dropped the trigger vol for Sunday, and the recording was fine. I left at ~3:30am this morning so as to pass through the Jasper park early enough to not have to pay $20 for 100k of road. I'm now in a lovely campground near Salmon Arm for a few days. They have wifi, so I'll be cutting code and soaking up the sun until Thursday.
Newest piece of tech around here is the Zoom H2 audio recorder which I picked up today. I'll be using it to record motion notion this weekend. It should work much better than last years failed laptop effort. Here's a brief overview:
The site is now being served using mod_proxy_ajp. This works better than the previous mod_proxy_http and mod_rewrite methods. I've described the integration of web applications with apache in the advanced section of the web tutorials. Subtle function of the site is now better.
I finished a very basic repository export and import application, which has allowed me to migrate object data from firebird to derby. It was a tricky little pig and is only just functional. It holds much promise, however. With a bit of effort it could more reliably move objects between databases supported by the object repository (firebird, derby and other jdbc databases). It has no key tracking or dependency checking now, but with a bit of manual sql massage, it was enough to move the site data into derby. The site is a bit faster, and the backend more up to date. The site is still, however, not as quick as I'd like it to be. I think the forum topic code slowing it down and might be a candidate for optimization... the whole thing should be cached anyway... but that's a bigger issue. The RepositoryIO migration tool is in the repository project.
I've installed Mantis Bug Tracker at bugs.benow.ca which will be used to manage bugs within the various benow projects. Sign up if you want to report or see changes.
A bought of the glitch has mostly been overcome. The menu is back, along with the projects. The codebase is making many strides, and this site was built on older iterations, so it's starting to become stale. I'm going to keep massaging it back to full function. It's still running on the firebird backend. Moving it to the jdbc derby backend should speed things up, but to do so, I need to finish the cross repository import/export code. Also, the method I'm using to map thru apache (mod_rewrite) is clobbering the server ajax error responses, but I think I can work around that with ajp13. The ipod/iphone formatting is fixed and no longer glitches (thx for the prod, Doug). Fun, fun, fun.
