ACZoom  ·  Archive-2016

Until recently I was happy with an old-school phone-only phone. Now I have some time on a phone-that-does-more-than-calls smartphone. It is a unlocked GSM Galaxy Nexus running Android 4.0 Ice Cream Sandwich.

First impressions are that this is great fun - a lot of opportunities to waste time of course. And it also can make and receive phone calls, but that seems like an minor side feature nowadays.

Unfortunately, the whole experience is not yet completely satisfactory. Lot of minor and major glitches abound, and it took a while to get some simple essential tools enabled.
My goal was to get the 16GB device to work mostly over Wi-Fi and use it as an offline MP3 music and .avi/mp4/flv video player. And to play long audio books well, with bookmarking capability.

Not so simple, it turns out!

Google Voice does not work over Wi-Fi

This does not work on the phone! Having been accustomed to using Google Voice on my computer, I expected a phone with no service (no SIM) should have Google Voice work over Wi-Fi. No luck. And a web search does not yield much info - no help at the Google Voice support pages certainly.
Some web pages do suggest that this is probably due to the legacy phone carriers imposing their will on Google. Maybe - but would be nice if this was clear on Google Voice pages, and it is still confusing that this holds true when there is no SIM - and no carrier involved at all.
In any case, the final answer is that Google Voice does not work on the phone over Wi-Fi. It must have a carrier voice and data plan.

Google Voice works great!

After over a year with Fedora 13, I updated my home desktop system to Ubuntu 11.10

Some 10+ years ago I hoped installing Linux would get easier over the years and one day I would be able to recommend it to non-tech family members. No such luck - getting ubuntu 11.10 up and running took too much time, and required too many difficult fixes.

My home machine runs a web browser. It is used for some minor video processing, GIMP image editing, digikam photo management, and is a host for a KVM/QEMU virtual machine that runs a web server for some specific tasks.

When installing a new operating system, I keep existing partitions so /home, etc is left unchanged. / is on its own, so can be completely cleaned and used by the new installer.
Most partitions are in a volume group, so LVM is necessary for booting.

1. First hurdle was downloading the installer, the Live CD. Pick the recommend 32-bit even though I (and all new computers in past 2+ years too) have 64-bit machines? Pick standard installer or dig into deeper links for the "alternate" installer? After a bunch of wrong downloads, and a lot of web searching - since Ubuntu site itself is not very helpful - determined that 64-bit install is just fine, and since I need to keep my existing partitions, the alternate installer is what I need and standard would not work (maybe!). Or at least that this combination would definitely do what I needed, and it was not clear if the standard install is good enough. So went with 64-bit alternate installer.
2. To run the alternate installer, I needed this information: info from old fstab and df to assign partitions. During the install, I formatted / but left /home and /data alone. These were all LVM volume groups. /boot was a physical partition. Also needed IP address info - the fixed IP address and gateway IP. For hostname, used name.localdomain (i.e. localdomain single word after hostname).
   

This was supposed to be easy - extending logical volumes. But if you install a virtual machine, then it all becomes a mess. Search the web for how to extend a partition "nested" in an LV, and there are only questions and no answers.

KVM Disk Management Issues shows an alternative to using the standard install - "just put a filesystem on it and you are done" which basically means that manual partitioning should be chosen during a Ubuntu install. Resize KVM Image shows another alternative which basically involves deleting the swap partition inside the KVM which allows the root partition to be enlarged. That would be necessary when there is no nested LVM, when the partitions were created in the hosts' logical volume. And for a general introduction to LVM see Logical Volume Management (IBM).

virt-manager makes it easy to install the virtual machine using an .iso image of the OS to install. It is not easy to resize storage on a KVM virtual machine, if installed following the standard instrutions - make a logical volume on the host machine, let the virtual machine installer use it like a raw disk to create its partitions, and install the OS.

Well, the trouble is now that the standard LVM resize procedures are not helpful. This is what the picture looks like, where VG is the host machine volume group:

Host Machine:

   vg:VG
    -- lv:kvm1
    -- lv:unused (lot of unused space on the volume group VG)

lv:kvm1 is the logical volume used to install the virtual machine. Let us say it is a Debian-based system, like Ubuntu. Using the default "Guided Partitioning", it will install a logical volume on the disk.

    -- lv:kvm1
       -- vg:VIRT
          -- lv:vroot     --- this is the partition we want to extend
          -- lv:vswap

This site uses Drupal. Drupal has turned into a nightmare. It was fine when there was a single 4.x version out there, but soon after 4.x, there was 5.x. Then 6.x. Upgrading from a older version is near impossible.

There always was the assumption that some amount of coding would be required by anyone running a Drupal site. But be prepared - you will be hacking modules left-and-right to get any thing running. At this time, one has to question whether the amount of hacking required to get things to run are worth it. Maybe all CMSes have this problem, but certainly Drupal is really a poster-child for impossible-to-ever-upgrade software.

The problem occurs because Drupal changes the API every release, adds new incompatible features, and modules and themes become unusable. And since modules and themes are merely someone's weekend project, it can be months or years before a module becomes compatible with the newer Drupal version.
Core drupal does not have image handling capabilities or spam fighting capability so even a basic site will need to use external modules. Add things like forums, automatic aliases, FAQs, it becomes a large collection of non-core modules.

The advantage of Drupal is that it is extensively customizable, and has a wide range of modules. This is exactly the same thing that makes a Drupal site near-impossible to upgrade. Once a site is up and starts to depend on a bunch of modules, rest assured that when a new Drupal version comes out quite a few required modules will not make it to that new version!

Drupal core does get upgraded without problems. But Drupal itself has become super-bloated. Web hosts that worked fine with Drupal 4.7 will not support Drupal 6.x because of heavily increased memory and CPU requirements.

dd-wrt is third-party firmware that can be loaded on many routers and it makes available many additional features such as advanced routing as well as a keep alive mechanism. It is maintained by BrainSlayer.

In the few days of using it, some advantages of dd-wrt are evident. It has been far easier to configure on my network of Linux and Windows computers, which use both static and DHCP IP addressing. The bundled Linksys software on the new WRT54G V8 device had long DNS lookup times on the Linux computers (probably needed to use the remote DNS resolvers instead of pointing to the Linksys box), for all lookups, at all times. But instead of re-configuring the Linux boxes, in the same amount of time, it was quite easy to install dd-wrt micro-edition using these instructions: How to Flash Linksys wrt54g

Notes on things to look out for working with Fedora Core installs.

perl CGI scripts hanging

Fedora Core 7 current perl package is perl-5.8.8-27 [Jan 2008]. This perl package contains version 3.15 of CGI.pm, which has a problem handling POST_MAX. Any HTTP post of size greater than POST_MAX value will cause the CGI script to peg the CPU at the $q = new CGI statement and not terminate for a very long time. After the CGI script is automatically terminated, a HTTP 500 Internal Sever Error status code returned to the requester.

This problem exists in all distributions using Perl 5.8.8, and is not limited to Fedora Core 7.

This issue is very hard to track down. Most of the time, the CGI scripts will work fine. When they do fail, there will invariably be nothing more than a single line in the Web server log stating that the job took around 300 seconds and a HTTP 500 Internal Server Error message was sent back. The times at which the failures occur will be all over the clock, there will be no pattern to the time when the failure occurs. If you do manage to see this rare failure happening, the CPU will show that all free CPU time is being used by the CGI script - but the script has no logging or database activity thus indicating that it locked up very early in the script. That will eventually lead to looking at the $q = new CGI statement, and to the fix.

The fix is simple - upgrade to CGI.pm version 3.21 or later. But to make this stick will require some additional configuration because while CGI.pm can be updated on its own, it is also bundled with perl. So to avoid a future yum update restoring CGI.pm back to version 3.15 when perl 5.8.8 is updated, requires preventing perl from being updated, or keeping two copies of CGI.pm around and changing scripts to load the right one.

A very small site started running into performance problems - some pages taking too long to load, and certain MySQL queries taking over 5000 to 6000 milliseconds, and being killed because of resource limits set on the hosting computer. The pages affected were the watchdog log display pages - one of which is the Menu -> administer page when logged in as the adminstrator, and it displays data from the watchdog table.

This seemed odd - for a site with less than 200 nodes, and very low traffic, there should be no performance issue, and no single database query should be taking as long as 6000 milliseconds.

So the options were to increase the time limit for queries, or to spend the time debugging the problem.

Drupal is very feature rich, and this may have negative impact on performance, but in this case, it turned out to be a database issue. The Drupal site has many performance related tips, including a subsection on Tuning MySQL for Drupal.

After looking around in the database for the site, it was discovered that the overhead for the watchdog table was over 40 times its actual size! So, the size was 176MiB, and the overhead was 172MiB. Running optimize on this table got the size down to under 4MiB, overhead to 0, and got the queries to be much faster - way below the 6000 millisecond time limit, and the administer and log display pages now rendered much faster, way below the old times.

One question remains - why did removing overhead fix the query times?

Time has never been easy to work with, and while it is best to use UTC to store time, it is necessary to use local time zones when displaying time to the user.

With regards to displaying local timezones using strftime and the format specifications %Z for timezone name and %z for the RFC-2822 conformant [+-]hhmm displays, Perl and PHP work fine, at least on Fedora FC5. Python 2.4 does not yet have support for %z, and the best support in Python for timezone is using the basic time module; the enhanced datetime module has no built-in support for time zones.

That takes care of strftime, what about strptime? Unfortunately, that is a topic that is even more convoluted, so try all variations out before you use it on any system.

Here's a strftime support summary for Python date/time users:

• Use the datetime module if you don't need any timezone handling. The standard library comes with no support for any timezones, not even local timezones, so datetime module is useless for those who need to use time zone information. This can be done of course, but requires coding up your own timezone routines.
• Use the time module if you can live with just %Z and don't need %z. Python 2.4 time module always prints wrong value +0000 for %z, even while it gets %Z correct.
• Need %z, the RFC-2822 conformant time display? This requires writing your own code, Python does not support this. Using the basic time module, here's example code on how to get these values:
  

  lt = localtime(t)
  if lt.tm_isdst > 0 and time.daylight:
      tz = time.tzname[1]
      utc_offset_minutes = - int(time.altzone/60)
  else:
      tz = time.tzname[0]
      utc_offset_minutes = - int(time.timezone/60)
  utc_offset_str = "%+03d%02d" % (utc_offset_minutes/60.0, utc_offset_minutes % 60)

  Note: in the utc_offset_str computation, the use of 60.0 float in the / operation is necessary to get a value rounded to 0 instead of negative infinity, for example, -90 minutes offset should be -0130 and not -0230.

Here's the sample code and the output from Perl, Php, and Python, for two example strftime calls:

Updating drupal using the standard instructions is very time consuming - have to turn off modules/themes, update settings, reinstall modules/themes.

But many users find that for many updates to the same major release, for example, 4.7.x series, simpler upgrades can work - official Drupal install instructions do not allow this, nor is the structure of Drupal folder structure a help, since user installed modules/themes are in the same folder as the drupal files (would be good to have these separate), and no easy way exists to try out a new release before switching to it on a running site.

Given all that, here's how an update can be done fast - very important to read the UPGRADE.txt Drupal document first, and do all backups, and be ready to restore quickly if things don't work.
No guarantees on this method, but it has been known to work.

Assume that old drupal install is in current/ and new one is new/

1. extract the new drupal release, into the new/ directory.
2. copy over all the new files and directories from your current install, to this new/ directory. See script "update.sh" below which does this.
3. update the config files - sites/default
4. login to current drupal as admin
5. backup: rename current/ to current.backup/
6. rename new/ dir to current/
7. run current/update.php from drupal and on success, log off the admin user. and test it out

And here's the update.sh script - edit $OLD variable, run this from inside the new/ directory, and redirect output to update.run, take a look at the update.run commands, and then run the commands in update.run:

#!/bin/sh
OLD="current/"

echo "# Current Directory: " `pwd`

for i in `cd $OLD; find . -print`
do
    if [ ! -e "$i" ]
    then
        if [ -d "$OLD/$i" ]
        then
            echo mkdir -p -v "$i"
        fi
        if [ -f "$OLD/$i" ]
        then
            echo cp -p -v -i --reply=no "$OLD/$i"  `dirname "$i"`
        fi
    fi
done

Fun with perl vs php regular expression handling.

PHP has a regular expression search and replace function, preg_replace. This is supposed to be the standard greedy algorithm for patterns, for example, A* means zero or more A characters, and it matches the longest string of A's at that point.

So, if you want to match zero or more slash (/) characters at end of a string the pattern to use is: /*$

And, if we want to replace zero or more slash characters at end of a string with a single slash character, the perl code is:   s!/*$!/!

Full example in perl:

foreach ('path/to', 'path/to//', 'path/to///', 'path/to////') {
    my $s = $_;
    $s =~ s!/*$!/!;
    print "string: $_ changed to $s\n";
}

Output is: 
string: path/to changed to path/to/
string: path/to// changed to path/to/
string: path/to/// changed to path/to/
string: path/to//// changed to path/to/

And the above output is all correct.

Equivalent PHP code does not work, using PHP version 5.1.6.

Following PHP code:  $t = preg_replace('!/*$!', '/', $s); fails, it will end up with one / if the input has 0 or 1 / characters, but it will end up with two / characters if the input has 2 or more / characters, instead of a single / character. So, the match was not greedy, but for reason, was split into two matches, and then each matched group was replaced with a single / character.
The workaround for this is to specify a limit count of 1 to preg_replace, which makes the code work fine. preg_match seems to work fine, only preg_replace has this problem.

Here's php code, and its output, showing the failure:

<?php
foreach (array('path/to', 'path/to//', 'path/to///', 'path/to////') as $s) {
     # preg_match('!/*$!', $s, $matches); // works fine, is greedy - $matches[0] is zero or all slashes
     # $t = preg_replace('!/*$!', '/', $s, 1); // works, limit of 1 helps