niels / Blog /

First Post!

First post on the new peen.dev domain. I’m expecting many more to follow.


niels / Code / #php

PHP TCP server

The first thing developed for Mailjoe was its TCP server. There are many examples out there, but we didn’t find any that was multi-protocol, non-blocking and multi-process.

The snippets below are fairly small and self explanatory. Please keep in mind that all of this code worked fine during the Mailjoe beta, but is by no means finished or necessarily suitable for other purposes.

mailjoe.php, the main loop:

set_time_limit (0);
require_once("config.php");
require_once(INCL_PATH.'Daemon.php');

$__daemon_listening = true;
$__daemon_childs    = 0;

declare(ticks = 1);

pcntl_signal(SIGTERM, 'daemon_sig_handler');
pcntl_signal(SIGINT,  'daemon_sig_handler');
pcntl_signal(SIGCHLD, 'daemon_sig_handler');
daemonize();

// Setup POP3 socket
$sockPOP = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_set_option($sockPOP, SOL_SOCKET, SO_REUSEADDR, 1);
socket_bind(		$sockPOP, $listen_address, $listen_port_pop3) or die('Could not bind to POP3 port/address!');
socket_listen(		$sockPOP, $max_clients);
socket_set_nonblock($sockPOP);

// Setup IMAP socket
$sockIMAP = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_set_option($sockIMAP, SOL_SOCKET, SO_REUSEADDR, 1);
socket_bind(		$sockIMAP, $listen_address, $listen_port_imap) or die('Could not bind to IMAP port/address!');
socket_listen(		$sockIMAP, $max_clients);
socket_set_nonblock($sockIMAP);

// Setup SMTP socket
$sockSMTP = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_set_option($sockSMTP, SOL_SOCKET, SO_REUSEADDR, 1);
socket_bind(		$sockSMTP, $listen_address, $listen_port_smtp) or die('Could not bind to SMTP port/address!');
socket_listen(		$sockSMTP, $max_clients);
socket_set_nonblock($sockSMTP);

// After we have privileged ports
posix_setuid($posix_uid);
function debug($s) {
syslog(LOG_INFO, "[".posix_getpid()."] ".$s);
}

while ($__daemon_listening) {

    $newsock = @socket_accept($sockIMAP);
    if ($newsock === false) {
    } elseif ($newsock > 0) {
        daemon_client($sockIMAP, $newsock, "imap");
    } else {
        die(socket_strerror($newsock));
    }

    $newsock = @socket_accept($sockPOP);
    if ($newsock === false) {
    } elseif ($newsock > 0) {
        debug("Accepted POP connection");
        daemon_client($sockPOP, $newsock, "pop");
    } else {
        die(socket_strerror($newsock));
    }
    
    $newsock = @socket_accept($sockSMTP);
    if ($newsock === false) {
    } elseif ($newsock > 0) {
        daemon_client($sockSMTP, $newsock, "smtp");
    } else {
        die(socket_strerror($newsock));
    }
    
    usleep(10000);
    
}

The relevant part from config.php:

define(BASE_PATH,'/home/mailjoe/');
define(INCL_PATH,BASE_PATH.'lib/');
define(TEMP_PATH,'/tmp/');

$listen_address		= '127.0.0.1';
$listen_port_pop3	= 110;
$listen_port_smtp	= 25;
$listen_port_imap	= 143;

As you can see, our process listens on localhost only. We were using Perdition to enable SSL as well as do some basic sanity checks for POP3 and IMAP4. SSLTunnel was used to enable SSL on the SMTP port.

We did initially use PHP’s built-in SSL capabilities but ran into a number of limitations.

daemon.php, which is based on several examples found on php.net:

function daemon_sig_handler($sig) {
    global $__daemon_childs;
    switch($sig) {
        case SIGTERM:
        case SIGINT:
            debug("SIGTERM received");
            exit();
            break;
        case SIGCHLD:
            debug("SIGCHLD received");
            while(pcntl_waitpid(-1, $status, WNOHANG)>0)
                $__daemon_childs--;
            debug("SIGCHLD finished");
            break;
    }
}

function daemon_client($sock, $newsock, $type) {
    global $__daemon_listening, $__daemon_childs;

    while($__daemon_childs >= MAX_CHILD) {
        usleep(10000);
    }
    
    $pid = pcntl_fork();

    if ($pid == -1) {
        debug("Failed to fork!");
        die();
    } elseif ($pid == 0) {
        $__daemon_listening = false;
        socket_close($sock);
        socket_set_block($newsock);
        $client = new $type($newsock);
        while($client->connected) {
            $sockets[0] = $newsock;
            socket_select($sockets, $write = null, $except = null, $tv_sec = NULL);
            $client->Check();
        }
        socket_close($newsock);
    } else {
        socket_close($newsock);
        $__daemon_childs++;
    }
}

function daemonize() {
    $pid = pcntl_fork();
    if ($pid == -1) {
        debug("Failed to fork!");
        exit();
    } elseif ($pid) {
        exit();
    } else {
        posix_setsid();
        chdir('/');
        umask(0);
        return posix_getpid();
    }
}

Important to note here is that daemon.php tries to create a new instance of $type, which could be POP3, SMTP or IMAP based on the socket used. You’ll have to put your own class here that can handle the connection.


niels / Software / #vpn

Linux L2TP/IPSec with iPhone and Mac OS/X clients

After moving to China I ran into a few issues that got me to install a VPN. Namely:

  • Latency. I frequently use SSH to access remote servers in US and EU and for some reason the latency here is terrible. Running SSH over a VPN seems to resolve this. A nice bonus is that idle sessions aren’t timed out (which they otherwise would with a little help of China’s Great Firewall.)
  • Location aware websites. A number of websites will adjust their content based on your location. This is fine if you’re able to adjust it manually, but many of these websites don’t allow this as they are trying to restrict content to US and/or EU users. (Which just so happens to be the content I’m looking for.)
  • Blocked websites. Although most websites that are blocked in China are not of particular interest to me, the GFW will occasionally put a ban on sites that I do like to use.

After using OpenVPN for a while I got mildly annoyed by:

  • High CPU usage (99%) on Ubuntu systems after running it for extended periods of time.
  • Not being able to use it on OS/X very easily or on my iPhone at all.

This made me switch to IPSec instead. IPSec requires a little more effort to configure but it has proven rock solid and cross-platform in several of my past projects. Having used Freeswan and Openswan before, I now decided to use Strongswan instead as it requires no kernel tweaks on Debian/Ubuntu making the installation a matter of minutes. (This is well documented on www.strongswan.org and I know you’re just here to read about the iPhone, so I won’t go into that now. 😉

Making IPSec work with iPhone and OS/X’s native clients requires installing an L2TP daemon.

First the fairly standard Strongswan configuration:

/etc/ipsec.conf:

config setup
# crlcheckinterval=600
# strictcrlpolicy=yes
# cachecrls=yes
nat_traversal=yes
charonstart=yes
plutostart=yes
conn L2TP
authby=psk
pfs=no
rekey=no
type=tunnel
esp=aes128-sha1
ike=aes128-sha-modp1024
left=your.ip.goes.here
leftnexthop=%defaultroute
leftprotoport=17/1701
right=%any
rightprotoport=17/%any
rightsubnetwithin=0.0.0.0/0
auto=add

/etc/ipsec.secrets:

your.ip.goes.here %any:     PSK "yoursharedkeygoeshere"

As you can see we’re enabling nat-traversal and transport mode at the same time. And that’s really the only ‘hack’ we need to do as it’s disabled by default.

On a Debian or Ubuntu system this should get you a long way:

apt-get install build-essential fakeroot dpkg-dev devscripts
apt-get source strongswan
apt-get install libcurl4-openssl-dev
apt-get build-dep strongswan
cd strongswan-4.2.4/
dch -i

Now edit debian/rules and change –disable-md5 –disable-sha1 –disable-sha2 to –disable-md5 –disable-sha1 –disable-sha2 –enable-nat-transport and continue:

dpkg-buildpackage -rfakeroot -uc -b
dpkg -i ../strongswan_4.2.4-5ubuntu2_i386.deb
/etc/init.d/ipsec restart

Alright. IPSec is good to go. Next the L2TP daemon for iPhone and OS/X:

apt-get install xl2tpd

/etc/xl2tpd/xl2tpd.conf:

[global]
debug network = yes
debug tunnel = yes
[lns default]
ip range = 10.0.0.200-10.0.0.254
local ip = 10.0.0.1
require chap = yes
refuse pap = yes
require authentication = yes
name = NIELSPEEN.COM
ppp debug = yes
pppoptfile = /etc/ppp/options.xl2tpd
length bit = yes

10.0.0.0/24 is your local LAN. 10.0.0.200-10.0.0.254 are IP addresses that we can freely assign to the users. 10.0.0.1 is a free IP on your local LAN. (It should not be the IP bound to your LAN interface!)

Note: you don’t need an actual LAN to make this work. In fact, the server I use the above config on is not connected to one.

/etc/xl2tpd/l2tp-secrets:

*       *       l2tppassworduser1 *

/etc/ppp/options.xl2tpd:

ipcp-accept-local
ipcp-accept-remote
ms-dns you.dns.ip.here
noccp
auth
crtscts
idle 1800
mtu 1410
mru 1410
nodefaultroute
debug
lock
proxyarp
connect-delay 5000

Great! Now to configure your iPhone:

  • Open the Settings app and navigate to General->Network->VPN->Add VPN Configuration…
  • Make sure the selected VPN type is L2TP. (Not IPSec.)
  • Fill in anything you like for Description.
  • Server should be the hostname or IP of the machine you just installed Strongswan and xl2tpd on.
  • Account is not checked by xl2tpd. You can give your users a random username.
  • RSA SecurID is to be switched OFF.
  • Password is the password as put in /etc/xl2tpd/l2tp-secrets.
  • Secret is the PSK you’ve put into /etc/ipsec.secrets.
  • Send All Traffic is typically turned on.

If, like me, you’re not using this to hook your iPhone to your office network, but want to use the connection to access the Internet, you’ll need to add a masquerading rule to iptables:

iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE

That’s it!


niels / Code / #php

Syncing Fanfou with Twitter

@kofai asked me to share my Fanfou to Twitter sync script. It’s not art, but it works, so here goes:

<?php
define(STATUS_PATH, '/home/www/twitbridg/status/');

include_once('password.php');
$since_id = file_get_contents(STATUS_PATH.$twitter_username.'.last') OR $since_id = '1';
// The twitter API address
$url = 'https://twitter.com/statuses/user_timeline/'.$twitter_username.'.xml?since_id='.$since_id;

$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL, "$url");
curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_GET, 1);
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);

if (empty($buffer))
  die('failed to get tweets: '.$since_id."\n");
$xml = new SimpleXMLElement($buffer);
$i = sizeof($xml->status)-1;

if($i<0) exit;

while($i >= 0) {
// Update FanFou
$url = 'http://api.fanfou.com/statuses/update.xml';
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL, "$url");
curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_POST, 1);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, "source=".urlencode("<a href=\"http://twitter.com/".$twitter_username."\">Twitter</a>")."&amp;status=".urlencode($xml->status[$i]->text));
curl_setopt($curl_handle, CURLOPT_USERPWD, "$fanfou_username:$fanfou_password");
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);
$i--;
}
$last_id = $xml->status[0]->id;
$fp = fopen(STATUS_PATH.$twitter_username.'.last', 'w');
fwrite($fp, $last_id);
fclose($fp);
?>
As you can see it includes a password.php. This sets a few login variables. You can just insert them at the top of the above PHP file, but as I edit my code in public places I prefer not to do that.

<?php
$twitter_username = 'nielspeen';
$fanfou_username = 'nielspeen';
$fanfou_password = '123xyz';
?>

Enjoy!


niels / Software /

RMVB files on (64-bit) Linux

The Chinese like to share their movies in RMVB (RealMedia) format as opposed to Xvid or DivX. Not sure why it got to be so popular (I would prefer the more open Xvid), but the file sizes and quality it produces are quite good.

A problem I ran into is that mplayer/vlc don’t seem to play these files without a bit of tinkering. I found a lot of how-to’s that tell you how to make it work, but non really straight forward on a 64-bit system.

Mplayer supports a number of codec libraries though, and more than one may get you result.

Where many how-to’s told me to install the RV40 codec (which I couldn’t find in 64-bit), I used the RV3040 that comes with Ubuntu by default instead. This fixes video.

For sound, the often mentioned solution was to install the w32codecs and use the cook codec. In my case however this resulted in choppy sound. Instead I use the ffcook codec (found in libmad) instead (also in Ubuntu by default.)

Finally my command line looks like:

mplayer -vc rv3040 -ac ffcook movie.rmvb

niels / Software /

AliPay on GNU/Linux

AliPay is the Paypal of China and hard to ignore if you want to use websites like TaoBao (the Chinese eBay.) Unlike Paypal however, it requires ActiveX components or Firefox plugins to function. Getting this to work on Linux can be a pain if you don’t read Chinese. So here’s in English:

Download http://blog.alipay.com/wp-content/2008/10/aliedit.tar.gz

Extract it into /usr/lib/mozilla/plugins (as root) or into ~/.mozilla/plugins (if you care about the current user only.) Restart Firefox and things should work! (Select Tools, Add-ons, Plugins to verify that Aliedit has been installed.)

Plugins directory may vary on other distributions.


niels / Software /

Google Gears 64-bit

Links to outdated and potentially dangerous binaries have been removed.

While eagerly installing Offline Gmail on my computers I ran into the fact the Google Gears is actually not available for 64-bit Linux. I managed to find a number of builds out there, but none that would actually work for me with Offline Gmail or the new WordPress.

Building it myself wasn’t as straight forward as I hoped, so let me safe you the hassle and allow me to share the result with you:

gears-linux-opt-05110.xpi

This build was tested with Firefox 3.0.5 on (k)Ubuntu 8.10 Intrepid. Enjoy!

EDIT: Firefox reports the download to be 4G, but it will complete successfully after 2.7M.

Updated binary:

gears-linux-opt-05180.xpi

This build was tested with Firefox 3.0.9 on (k)Ubuntu 9.04 Jaunty. (It will install but NOT work on Firefox 3.5.)

Updated binary:

gears-linux-opt-05210.xpi

Did not change anything for Firefox 3.5 this time, since it doesn’t work anyway.