Paralells Virtuozzo Linux 4.7 kernel panic

After upgrading from PVCfL 4.0 / 4.6 to version 4.7, kernel panics started to occur randomly on almost all hardware nodes.

I configured kdump on the nodes to collect a kernel dump upon the next kernel panic, and contacted Parallels support team.

When the kernel panic occured and the dump was created, they analyzed it.

The feedback we got was the following:

The latest saved dump points to general protection fault caught by `mysqld`. This looks like just recently found issue - PCLIN-30321. I will pass it to the maintenance team.

As for the type of installation - there is no difference if this is a clean installation or an upgrade from the previous version of Virtuozzo.

The issue is quite complex (General Protection Fault from a process in a container is quite difficult to investigate), and it would take some time for maintenance / development team to check the case.

Though, the investigation might take significant amount of time and therefore the results will be provided through the separate ticket once they are ready.
You can also check the release notes for Virtuozzo kernel updates (CU-2.6.32-042stabXXX), and monitor the status of the request PCLIN-30321 which was assigned for this crash issue.

It's not always caused by 'mysqld', I've seen a kernel panic caused by another process as well, but most were indeed due to mysqld.
We don't know how long it will take for this to be fixed, so we decided to migrate the containers which cause the kernel panics to a 4.0 or 4.6 node.

Finding out which container caused the kernel panic can be done by using the crash tool.
An example:

[root@virtuozzo 127.0.0.1-2011-11-05-08:00:17]# crash /root/vmlinux-2.6.32-042stab037.1 vmcore
crash 4.1.2-8.el5.centos

      KERNEL: /root/vmlinux-2.6.32-042stab037.1
    DUMPFILE: vmcore
        CPUS: 8
        DATE: Sat Nov  5 10:00:13 2011
      UPTIME: 10 days, 10:12:02
LOAD AVERAGE: 19.68, 15.35, 14.01
       TASKS: 2239
    NODENAME: virtuozzo
     RELEASE: 2.6.32-042stab037.1
     VERSION: #1 SMP Fri Sep 16 22:18:06 MSD 2011
     MACHINE: x86_64  (1995 Mhz)
      MEMORY: 32 GB
       PANIC: ""
         PID: 62872
     COMMAND: "mysqld"
        TASK: ffff88024d6892c0  [THREAD_INFO: ffff8802d65e4000]
         CPU: 5
       STATE: TASK_RUNNING (PANIC)

crash> dmesg
[900722.171220] Process mysqld (pid: 62872, veid=7420232, threadinfo ffff8802d65e4000, task ffff88024d6892c0)

Update 15/11/2011
On 14/11/2011, Parallels has released a kernel update which contains a fix for this problem. More information on http://kb.parallels.com/en/112809

Plesk 10: display os/browser/robots in awstats statistics

I'm using Plesk 10.1 on a Debian 5.0 VPS, and noticed that the statistics (Awstats) for my domains did not contain all the information I want, such as statistics for operating systems, browsers, robots/spiders.

I checked several configuration files, and found that the "plesklog" format, that Plesk uses to define the Apache LogFormat value, does store this information in the logfiles.
However, the Awstats model configuration file is not properly configured to retrieve this information from the logs.

By default, the value of the LogFormat parameter in /etc/awstats/awstats.model.conf will be "LogFormat=4", which is the Apache native common log format, not the combined, or plesklog format.
By changing this value to "LogFormat=1", these features (browsers, os, robots,...) will be shown in the statistics of your domain(s).

After modifying the awstats.model.conf file, you'll need to modify the awstats configuration files for your existing domains.
You can use this perl script to do so:
perl -e "s/LogFormat=4/LogFormat=1/g;" -pi $(find /opt/psa/etc/awstats/ -type f)

This will go through all the files inside the /opt/psa/etc/awstats/ directory, look for "LogFormat=4", and replace it by "LogFormat=1". Be sure to create a backup of all your files, in case something goes wrong.

The new statistics features will only be visible after the next calculation.

Parallels Virtuozzo Linux vzbackup error

Last week, I installed a new PVC Linux 4.6 node, and added it as a slave to an existing PVA (Parallels Virtual Automation) group.
I use the vzbackup cronjob to backup all the containers on the hardware nodes.
The node that initiates the backup cronjob, had PVC Linux 4.0 installed, all the other slave nodes have version 4.0 installed as well.

The backup of the new 4.6 node failed with this error message:

vzbackup(2861): Starting backup. Nodes - virtuozzo10
vzbackup(2861): Starting node virtuozzo10 backup...
vzbackup(2861): Checking backup version on virtuozzo10 ... use vzbackup 4.6.0-9
vzbackup(2861): Starting node virtuozzo10 backup...
vzbackup(2861): Starting backup CT 1(virtuozzo10)...
bash: /usr/share/vzbackup-4.6.0-9.swsoft/vzbackup1: No such file or directory
vzbackup(2861): Can not read layout version for CT 1(virtuozzo10)
vzbackup(2861): No containers intended for backup found on virtuozzo10

The cause of this error message is that vzbackup 4.0 cannot create backups of containers on a 4.6 node.

Luckily, the other way around does work. After upgrading the node that initiates backups from 4.0 to 4.6, backups of the new node succeeded.
Backups of the other 4.0 nodes also remained successfully.

Using vzbackup on a PVC 4.6 node to backup 4.0 nodes is a supported method by Parallels.