Ubuntu DVD

Playing DVD's in Ubuntu isn't too difficult (see the Ubuntu doc or howto). I got good results from VLC. From there k9 can make an ISO of a movie if you need to back it up. Gnome can then burn the ISO to a DVD+R.

alpine

I can use pine again! Sort of. With alpine I can use a pine-like mail client and not have to worry about the undesirable license. Thanks slashdot.

vi query replace

I believe that emacs is ultimately more powerful than vi because it is written in Lisp and allows you to modify how it works on the fly. However, vi is smaller and faster -- sort of for the same reason -- and I prefer to use it for quick edits. In emacs I often M-% to query replace. So it's about time I found out how to do the same thing in vi:

: %s/foo/bar/gc

It's a lot like sed except for the c option which sed doesn't have since it's non-interactive by definition:

$ echo "I like unix" | sed s/i/u/
I luke unix
$ echo "I like unix" | sed s/i/u/g
I luke unux
$ echo "I like unix" | sed s/i/u/gi
u luke unux
$ echo "I like unix" | sed s/i/u/gc
sed: -e expression #1, char 8: unknown option to `s'
$

Think of the c as confirm.

https overview

I. If you're making your own numbers:

You just need a certificate and a private key. These can be encoded in one PEM file. It's easy to make your own PEM file with your own self-signed cert on a vanilla RedHat apache:

cd /etc/httpd/conf/
make host.domain.tld.pem
mv host.domain.tld.pem ssl.crt/

Just be sure to edit ssl.conf to reference the new cert. You can replace:

 SSLCertificateFile /etc/httpd/conf/ssl.crt/server.crt
 SSLCertificateKeyFile /etc/httpd/conf/ssl.key/server.key

With just:

 SSLCertificateFile /etc/httpd/conf/ssl.crt/host.domain.tld.pem

and then restart apache.

II. If you're buying numbers:

You'll probably be asked to:

1. Generate a .key
2. Generate a .csr
3. Give the CSR to the company and get back a .crt

Then have your httpd.conf reference the above respectively with the mod_ssl directives:

1. SSLCertificateKeyFile /usr/local/ssl/private/verisign.key
2. N/A
3. SSLCertificateFile /usr/local/ssl/certs/cert.crt

Once you have your cert you can look it over and see relevant things like how long it will be valid for with:

openssl x509 -noout -text -in cert.crt

Note that you might also run into intermediate certificates. Which Apache can reference with either of the following directives:

 SSLCertificateChainFile /usr/local/ssl/certs/intermediate.crt
 SSLCACertificateFile /usr/local/ssl/certs/intermediate.crt

depending on your version of Apache.

Duke NFS Slides

I found some nice slides on NFS from Duke.

name your server

Someone where I work wanted a bad name for a host. Part of it ended in ge since it had gigabit ethernet. We came up with a better name. I found a post on this topic. I think sprint has the right idea.

sync; sync; halt

Several links on the controversial: sync sync halt.

One even claimed that sync on recent Linux kernels appears to be synchronous and not return until the data is out the door. I thought that only fsync did that.

MPIO

I'm considering native Multi Path I/O (MPIO) as an alternative to EMC PowerPath 5 for stability reasons. MPIO has been native since the 2.6.13 kernel. Back ports have been in RHEL since version 4U2. Dell has an overview and the multipath-tools project page's ReferenceBook has more details. The 2005 Linux Symposium (pages 155-175) also covers it and calivia.com has test results.

sync

File systems don't write your data to the disk directly. They buffer it until later and then write it. Thus, block level snapshots of your disk can be incomplete.

Sync can be used to insure file system integrity, but it buffers a message to write to disk instead of waiting to return until everything is written to disk. The fsync system call takes a file descriptor as an argument and won't return until data is written to disk. Chapters 3.14 and 4.24 of APUE cover this in more detail. EMC seems to advise (page 10) that you sync and umount a disk before using their block-level SANcopy tool.

doodle

This scheduling tool works well to identify the best date(s) for meetings: http://www.doodle.ch/main.html

python primes

#!/usr/bin/env python
# Filename:                primes.py
# Description:             Returns all primes less than N
# Supported Langauge(s):   Python 2.5.x
# Time-stamp:              <2007-12-10 00:38:14> 
# -------------------------------------------------------
# I. Find all primes <100 using the Sieve of Eratosthenes:
# 
# By the fundamental theorem of arithmetic: 
#  * every postive integer is a prime xor a composite
#  * every composite can be written as the product of primes 
#  * if n is a composite, then n has a prime divisor <= sqrt(n)
#
# So we'll start with a list of primes whose prime factor is less
# than sqrt(100), or 10.  These primes will be 2,3,5,7.
# 
# To use the sieve we do the following:  
# 
# * integers divisible by 2 (besides 2) are deleted
# 
# * since 3 is the first integer greater than 2 that is left,  
#   all integers divisible by 3 (besides 3) are deleted
# 
# * since 5 is the next integer after 3 that is left, all integers
#   divisible by 5 (besides 5) are deleted
# 
# * since 7 is the next integer after 5 that is left, all integers
#   divisible by 7 (besides 7) are deleted
# 
# * since all composite integers not exceeding 100 are divisible
#   by 2,3,5,7 all remaining integers except 1 are prime.  
# 
# Runtime:  n * sqrt(n)
# -------------------------------------------------------

from math import sqrt
from math import ceil

def divides(a, b):
    return a % b == 0

def sprimes(n):
    integers = []  
    for i in range(2, n): 
        integers.append(i)

    max_factor = int(ceil(sqrt(n))) 
    factors = integers[:max_factor] 

    for factor in factors:
        for integer in integers:
            if (divides(integer, factor) and (integer != factor)):
                integers.remove(integer)
                if (integer in factors):
                    factors.remove(integer)
    return integers 

print sprimes(1000)

# -------------------------------------------------------
# II. find all primes <100 using Wilson's theorem:
# 
#  p>1 is a prime iff fact(p-1) == -1 (mod p)
# 
# As per:  http://en.wikipedia.org/wiki/Wilson's_theorem
# 
# Translating modular arithmetic into Python:
#  http://www.muppetlabs.com/~breadbox/txt/rsa.html#3
# 
# As the above states "27 == 3 (mod 12)" in Math would be 
# "27 % 12 == 3" in Python.  Taking it to the next step:
# "fact(p-1) == -1 (mod p)" would be "fact(p-1) % p == -1".
# However, since a programming language's modulo returns an
# integer greater than or equal to zero I won't get a -1 so
# I'll use "fact(n-1) % n == n-1".  

def fact(n):
    f = 1
    while (n > 0):
        f = f * n;
        n = n - 1;
    return f;

def prime(n):
    return (fact(n-1) % n) == (n-1)

def wprimes(n):
    integers = []  
    for i in range(2, n):
        if prime(i):
            integers.append(i)
    return integers

print wprimes(1000)
        
# -------------------------------------------------------
# The Sieve of Eratosthenes ran faster than Wilson's theorem,
# probably because of the factorial (try it on 1000)

topcoder

I registered with topcoder this weekend. It might be fun to keep my wits sharp with the occasional algorithm competition. I tried one of the practice problems. I like Java more than the other languages to solve the problem. The problem didn't seem to demand object orientation so I ended up using static methods. This was a problem (An illegal access exception occurred. Make sure your method is declared to be public) when I tried to submit my work back into their system since they seem to instantiate your class and then call your method to test. I think I found a bug or at least in inconsistency with the question. I reported it so we'll see. Also, I looked over the problem and didn't come back to it until the next day. This probably made it take too long and I think that's why I only got ~90 points. My solution seemed to work for all of their cases. Better luck next time.

Update: Topcoder acknowledged the bug in the 300pt 1-SRM 144 Div 1 problem. The statement considers example "123210122", so the second line you quote should look like

Because Q[0] = P[0] + P[1] = 1 + P[1] = 1 (!!), we know that P[1] = 0.

instead of

Because Q[0] = P[0] + P[1] = 1 + P[1] = 0, we know that P[1] = 0.

Java Stack Trace

I'm messing around with Java. Ubuntu has a write up. I choose IcedTea:

apt-get install icedtea-java7-jdk

I also installed mmake. I'm revisiting the Knock Knock server. Doing a SIGQUIT kill (-3) causes it to dump core and keep running:

$ java KKMultiServer > log 
...
$ ps axu | grep java | grep -v grep
user  24050  0.5  0.3 1308620 12740 pts/22  Sl+  15:24   0:00 java KKMultiServer
user  24071  2.0  0.3 1377796 15024 pts/20  Sl+  15:25   0:00 java KnockKnockClient
user  24093  2.4  0.3 1378512 15456 pts/21  Sl+  15:25   0:00 java KnockKnockClient
user  24113  4.3  0.3 1377708 15472 pts/19  Sl+  15:25   0:00 java KnockKnockClient
$ kill -3 24050
$ ps axu | grep 24050 | grep -v grep 
user  24050  0.0  0.3 1308620 12788 pts/22  Sl+  15:24   0:00 java KKMultiServer
$ wc -l log 
117 log

The stack trace output is interesting. It seemed to list each thread that the KKMultiServer had:

"KKMultiServerThread" prio=10 tid=0x00000000006e8400 nid=0x5e43 runnable [0x0000000041412000..0x0000000041412d90]
...
"KKMultiServerThread" prio=10 tid=0x00000000006e6400 nid=0x5e2f runnable [0x0000000041311000..0x0000000041311c10]
...
"KKMultiServerThread" prio=10 tid=0x00000000006d2400 nid=0x5e1b runnable [0x0000000041210000..0x0000000041210c90]

Then I see other runnable threads like the Low Memory Detector, Compiler Thread, Signal Dispatcher, Finalizer, Reference Handler, main, VM Thread and GC tasks. Each was waiting or runnable. It also prints Heap information. I came across PSYoungGen.java when trying to make sense of it. I found two stack trace articles from Sun and [0xCAFEFEED]. It's nice to see that people find this useful.

amd-v xen

I'm looking to virtualize some services with xen and since I'm buying new hardware I'm curious about the amd-v chip.

I glanced over an AMD paper on xen on the amd-v chip. I also found a blog post which discusses these types of chips and the my hypervisor uses cpu hardware extensions to do what you do in software so it's faster than yours debate. The first benefit you'll hear about from AMD-V seems to be allowing unmodified guests to run on xen. I.e. the OS that you want to host doesn't need to be ported to xen. Since I'm mainly looking at running Linux I'm not too excited. The paper also claims it "reduces overhead by selectively intercepting instructions destined for guest environments". This seems plausible, though I'm curious if it's just newspeak.

AMD also claims that the memory controller "provides efficient isolation of virtual machine memory". This seems to be the most relevant benefit for me. I don't know enough about memory controllers to fully appreciate this. So far I'm hearing about memory virtualization (i.e. AMD Nested Page Tables and Intel Extended Page Tables) but virtal memory is nothing new. The difference seems to be a focus on isolating memory for a virtual machine as noted by the inquirer and project-xen.web.

Unbuffered I/O System Calls (apue ch3)

apue ch3 covers Unbuffered File I/O System Calls. It starts with:

• open takes a path and returns a file descriptor (or -1 on error)
• The kernel hands a file descriptor int to a process which reads or writes to it
• creat is equivalent to open(path, O_WRONLY | O_CREAT | O_TRUNC, mode)
• close(fd) releases process locks on a file and returns 0 (or -1)
• Every open file has a current offset --stored in kernel at no I/O cost-- initialized to 0 unless O_APPEND is used
• lseek (fd, off_t, whence) returns the offset (or -1)
• whence can be: SEEK_SET (off_t from beginning) SEEK_CUR (off_t from current) SEEK_END (off_t + file size)
• currpos = lseek(fd, 0, SEEK_CUR); i.e. the new file offset zero bytes from current is a goodway to get the current offset
• write(fd, *buff, n_bytes) writes n_bytes of *buff to fd, increments offset, returns n_bytes or -1
• read(fd, *buff, n_bytes) reads n_bytes of *buff from fd, increments offset, returns n_bytes or -1
• unlink(path) deletes a file
• chmod(path, mode) changes file permissions
• stat(path, struct stat *sb) returns ptr to stat struc with file inode info

Note that the last three sys calls above are not in ch3 of apue and that I borrowed them from File-Related System Calls in FreeBSD.

With this in mind we can "create a hole in a file". I.e. the filesystem just pretends that at a particular place in the file there is zero bytes, but no actual disk sectors are used. When this happens the offset is still incremented so that future writes don't fill the hole. E.g. hole.c:

if ( (fd = creat("file.hole", FILE_MODE)) < 0)
   err_sys("creat error");
// create file.hole in current directory

if (write(fd, buf1, 10) != 10)
 err_sys("buf1 write error");
// write 10 bytes of buf1 to fd (offset = 10)

if (lseek(fd, 40, SEEK_SET) == -1)
 err_sys("lseek error");
// seek 40 bytes in memory from beginning (offset = 40) 

if (write(fd, buf2, 10) != 10)
 err_sys("buf2 write error");
// write 10 bytes of buf2 to fd (offset = 50)

Note that ls reports that file.hole is 50 bytes. We can then see the holes with od:

:~/code/c/apue/ch3> ll
total 20K
-rw-r--r-- 1 anonymous anonymous  542 2007-12-05 23:42 hole.c
-rwxr-xr-x 1 anonymous anonymous 8.5K 2007-12-05 23:42 a.out*
-rw-r--r-- 1 anonymous anonymous   50 2007-12-05 23:56 file.hole
f:~/code/c/apue/ch3> od -c file.hole 
0000000   a   b   c   d   e   f   g   h   i   j  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000040  \0  \0  \0  \0  \0  \0  \0  \0   A   B   C   D   E   F   G   H
0000060   I   J
0000062
:~/code/c/apue/ch3> 

Ch3 then provides an unbuffered cat which differs from K&R's Ch1.5.1 cat:

#include <stdio.h>
/* copy input to output; 2nd version  */
main()
{
    int c;
    while ((c = getchar()) != EOF)
        putchar(c);
}

Which uses Standard Buffered I/O and is covered later in apue ch5.

The rest of the chapter covers File Sharing Between Processes, Atomic Operations and ioctl.

apue init

I'm finally trying some example programs from apue. When I try to compile hole.c with ourhdr.h in the same directory I see an "undefined reference to `err_sys'" error. This makes sense since I haven't defined the error functions. One solution is to replace them with printf. Since the apue site has error.c it's easy enough to just include that file. I'm working in a subdirectory I have for apue and with some slight modifications to hole.c:

:~/code/c/apue/ch3> diff hole.c hole.c.orig 
4,5c4
< #include      "../ourhdr.h"
< #include      "../error.c"
---
> #include      "ourhdr.h"
:~/code/c/apue/ch3>

I've got the example working on my Ubuntu system.

CUPS Export

You can export your CUPS settings as an XML file and then import them to another system. You can't just scp the /etc/cups/printers.conf file since it won't contain all of your printing configuration.

up2date has no dist-upgrade

With Debian/Ubuntu you can "apt-get dist-upgrade" but I know of no equivalent in RedHat. According to the RHEL5 release notes, you may "perform an upgrade from the latest updated version of Red Hat Enterprise Linux 4 to Red Hat Enterprise Linux 5". But they don't say how. MIT has some instructions which propose either having the installer upgrade or doing a fresh install. There also seem to be some yum issues after the upgrade.

LUNz

LUNz is created when a host has connectivity to an array, but no LUNs have been assigned to it. This allows the host to see the array and make use of a very limited set of SCSI commands. There are more or less technical explanations.

You can see if you have a LUNz device by looking for "Z" items in /proc/scsi/scsi. The last two entries below are LUNz's:

$ cat /proc/scsi/scsi
Attached devices:
...
Host: scsi1 Channel: 00 Id: 01 Lun: 00
Vendor: DGC Model: RAID 5 Rev: 0324
Type: Direct-Access ANSI SCSI revision: 04

Host: scsi1 Channel: 00 Id: 02 Lun: 00
Vendor: DGC Model: LUNZ Rev: 0324
Type: Direct-Access ANSI SCSI revision: 04

Host: scsi1 Channel: 00 Id: 03 Lun: 00
Vendor: DGC Model: LUNZ Rev: 0324
Type: Direct-Access ANSI SCSI revision: 04
...
$ 

If you've connected a host to a second SAN but haven't assigned any real LUNs to it then an SP Collect will show you port number of the Second SAN's Service Processor which is most likely a LUNz.

It's been reported that there can be blocked I/O to EMC CLARiiON LUNz paths.

You can get rid of a LUNz by removing the zone that connects your host to the particular SAN with the LUNz OR you could add a "real" LUN to that SAN. You'll then want to reload the HBA module since it allows the kernel to see the the SCSI bus in it's final state. Rebooting is recommended to be on the safe side.

xubuntu latex

LaTeX on xubuntu is easy:

apt-get install tetex-base tetex-bin tetex-extra

Then I can use the Ultimate Latex Makefile to easily "make pdf" and "make clean". When making updates I tend to "make foo.pdf && xpdf foo.pdf"