Parallel computing and communication/network models - the "parallels"

Apologies for the pun, but one of my readings over the last week has been to read up on parallel computing in order to find some parallels between the opportunistic applications I have been working on (such as 7DS) and parallel computing.

Even though the two topics look different, there are actually several points in common.

Below is a list of topics, extracted from the textbook Parallel & Distributed Computing Handbook by Albert H. Zomaya, published by McGraw-Hill.

• PRAM (Parallel RAM) model; Methods for distributing and parallelizing algorithms
• Failures in parallel systems (Byzantine, other)
• Dataflow models
• Checkpointing (for recovery)
• Distributed software systems (this is similar to RMI)
• Data structures for parallel processing
• Shared memory, memory consistency

More details to come soon.

Finished BibTex on IRT publications page

Today I fixed up the BibTeX links on the IRT publications page. I also changed the links so they open in a new, smaller window (like a dialog.)

Papers related to communication/networking middleware: IEEE Network

Am on the trail of papers related to work on communications or networking middleware from a high-level perspective. Been reading IEEE Network publication online all day, and found some interesting issues as well as articles that may be quite relevant:

• (2008) Vol 22, Issue 4: Context aware networks, in mobile scenarios (whole issue)
  
• (2007) Vol 21, Issue 4: GE Network Systems Architecture (whole issue)
• (2006) Vol 20, Issue 4: VON: a scalable peer-to-peer network for virtual environments (this paper has analysis of other P2P-based NVE systems)
• (2006) Vol 20, Issue 3: A survey of transport protocols for wireless sensor networks
• (2004) Vol 18, Issue 1: GE Middleware technologies for future communication networks (whole issue, and is very good)

Among other things:

• Registered and uploading final version of paper for CCNC 2009 conference.
• Testing the new WORKIT box, which may be the final version of the box we will ship
  

Grid computing and P2P resource discovery

Excellent survey paper on various programming models in grid computing, which relates very closely to what I am looking into in finding programming models for the get/set and distributed network computing environment at large: "Grid Programming Models: Current Tools, Issues and Directions".

Another related work paper is "Peer-to-Peer Resource Discovery in Mobile Grids," which defines a P2P service discovery protocol/framework for mobile environments.

One other direction I am currently looking into is the relationship between grid computing, cloud computing and the mobile distributed computing I am working on...

Network programming: scalable web servers and Java NIO

While looking for network programming APIs for my candidacy, I came across two excellent papers/presentations which are not directly related to my topic, but I need to bookmark in any case...

The first one is Scalable Network Programming: Or: The Quest For A Good Web Server (That Survives Slashdot) by Felix von Leitner. It talks about the problems of building scalable web servers, from ground up, including looking at fork() code, the time involved, measurements, comparison with threads, etc. An excellent read for people who know scaling web applications is hard, but would like to know more of why from an "under the hood" approach.

The second one, Improving Java Network Programming, by Brian Runk, is a simple application-level presentation, but compares the java.net and java.nio packages and discusses practical issues that arise while programming large-scale enterprise systems like ones built at Morgan Stanley.

Sockets over HTTP: Comet and Orbited

After my talk on "Disruption Tolerant Applications in Opportunistic Networks" at the IRT meeting yesterday, I found some interesting info about sockets over HTTP using Javascript.

The Javascript implementation, called Orbited, claims to provide a TCP socket over a web browser. Sounds like a really interesting concept. A slightly outdated tutorial on writing an IRC client using Orbited.

This is related to a neologism called Comet, which is "a web application model in which a long-held HTTP request allows a web server to push data to a browser, without the browser explicitly requesting it." (quote from Wikipedia's entry on Comet.)

Related work: Market Contact Protocol

An interesting Master's thesis called "Content Sharing for Mobile Devices" has an extremely comprehensive and complete look at the problem of finding and sharing content with other local mobile devices.

The thesis has excellent documentation comparing DHTs, overlay networks and other related work in this field.

The simulation and datasets in the thesis are also very comprehensive, and may be helpful if we ever do a simulation ourselves in this field.

The implementation itself, a protocol called "Market Contact Protocol" - at first glance - seems very similar to what we are doing with the BonAHA project, and related work such as LightPeers.

Nokia n800, Python, Bonjour and mDNS

A project student has the task of developing useful 7DS-like applications on our lab's Nokia n800, so we went searching for how to develop applications on this platform.

It appears that Python is the Nokia n800's programming language of choice - tough luck for Java. So we would have to start working with Python and developing our framework in that language.

In order to use Bonjour on Python for the Nokia, we need the pyBonjour library.

We also need to install the Bonjour mDNSd daemon for handling mDNS packets - or Avahi, which is more supported on Linux. A look around reveals that Avahi is available on the Nokia n800 through the Canola project.

But for pyBonjour to work with Avahi, the binary-compatibility layer with Bonjour need to be installed [libdnssd1 and libdnssd-dev].

There are existing demo applications (one and two) on Nokia using Bonjour. Both seem to relate to web servers.

Some more info about Python and mDNS from razorvine.

Data-object vs communication P2P applications

It appears that there are two sets of applications that BonAHA (or any ad-hoc network framework) will have to cover and address:

• Data-object and file-oriented: In this sort of application, the most important topic are the files and the data themselves, not necessarily the location of these. E.g.: file sharing, file synchronization, chat
• Person-to-person communication: In this sort of app, the focus is on the person to communicate with (not even the node itself, which is an abstraction, but the person). For example: drag-and-drop file sharing, instant message

So, BonAHA will have to address which of these two models it will be targetting - and if it can, and will, target both classes of applications.

DHTs actually handle the data-object model very well, expect that their setting up of an overlay network and additional network traffic in networks with heavy churn leaves much to be desired for MANET applications.

mDNS and DHT-related work

This was a very productive week in looking forward to working more on the BonAHA framework and our architecture for disconnected ad-hoc networks.

• mDNS and DNS-SD: Reading the specifications again, particularly focusing on their handling of:
  
• Packet size: How does it deal with large packets? What is the maximum packet size?
• iChat TXT record data: iChat exchanges its avatars using DNS-SD. But how large is it? What are the tradeoffs?
  

In terms of related work, here are some items I looked at this week that seem related to 7DS and the BonAHA framework:

• P2PSIP RELOAD
  
• System on Mobile Devices, SyD (A paper from this got accepted at Usenix 2004)
• LIMO: But this doesn't actually seem to have much relationship to 7DS/BonAHA work - it seems to be an overall, comprehensive
  

Updated website + Fall courses

As Fall semester rolls around, I decided to start it off by updating my website to use a new template - Concrete on FreeCSSTemplates.org. (PS: FreeCSSTemplates.org has some great, simple and totally free CSS templates for your website.)

I will end up taking "Analysis of Algorithms, I" this semester and may end up auditing the "Artificial Intelligence" course if possible...

WORKIT website now detects duplicate usernames; gives error messages

The WORKIT website now gives error messages properly for two common reasons:

1. Duplicate username
2. Empty fields

I have put up the code here since Blogger's code highlighting system isn't too good.

Change to a new DB server

1. Tis the day the WORKIT website moved to a new DB server, the shared MySQL server ... so we don't intrude on department MySQL server space. Requires a bit of recoding on my part to get the WORKIT website working with the new server.
   
2. Also, found that users can create duplicate entries with same username - that's a no-no. Need to fix this.
3. The Physics Machine Shop and I played phone tag today - we finally decided to meet on Thursday to finalize the hardware setup for the remaining baseline kits.

vi / vim extra tabs during paste - solved

I've had this annoying problem with vi / vim ever since I started to use it - whenever you copy text and paste it into vi or vim, it indents each additional line with an extra tab, rendering it completely unreadable and requiring manual editing.

Luckily, I was able to find help on Google groups and the Wikia Vim pages. Solving this problem is as easy as typing in the command:

:set paste

That worked for me!

Wireless cards, antennas and connectors

Today I hunted down a wireless antenna-cable combo to go with our wireless card, the Atheros-based Gigabyte GN-WIAG02.

I didn't realize that there were different forms of cables and antenna for wireless cards, so I had to learn more about these.

I found out on Hyper Link Tech that there are several types of these. Because the Gigabyte card is a 802.11 b/g, which runs on the 2.4GHz frequency, what I needed was a 2.4 GHz rubber duck antenna. 5 dBi seems to be fine for the WORKIT testbed.

There were some helpful pages on gumph.org and Seattle Wireless that helped me find out more about the antenna, cables and connectors that are needed for wireless.

It appears that the most common connector for the miniPCI based wireless cards is the MMCX connector.

Finally decided to get the AOpen wireless antenna, which appears to have the antenna as well as the cables needed to fit the antenna onto the kit.

Rich UIs in Java: themes and fade animations

I was looking for ways to improve the look and feel of Java applications - for presenting my Java-based research work as a nice-looking app and for demos. I was inspired by some of the Adobe AIR themes out there, and I found some interesting stuff.

Among the many Swing Pluggable Look-and-Feel solutions is one that really stands out - Substance. It looks absolutely phenomenal, with a variety of options and customizations, plus its open-source. It easily stands out as a must-use PLAF for Swing. Take a look at some Substance screenshots.

There's that for themes ... but how about fade effects and transitions?

A little bit more searching reveals the animation layer, which includes transitions - in fact, automated transitions with no changes to app code! You need to see the above link for yourself to believe it.

Apparently, this fade animations is part of the laf-widget project now.

insmod, kernel versions, modules and Linux distros

The joy of working with kernel versions and kernel modules.

So after spending a long time finally getting a modpost program running so I could build kernel modules, I was finally able to get them to compile. However, I found that the modules were not loading - insmod and modprobe gave a very unhelpful "-1 Invalid module format" error.

A little bit of searching around for "kernel modules" lead me to the helpful Linux Loadable Kernel Module HOWTO. A page titled Technical details provides the full details about how to find information about the module.

I was able to confirm that the module had compiled properly, and that it was atleast displaying the right meta-information. So what had gone wrong?

A search for the actual error message "invalid module format" led me to a forum discussion on Linuxquestions.org about insmod, where one poster suggesting using 'dmesg' to find out the details of the error.

Excellent information. A line in dmesg shows:

No module found in object
kaodv: version magic '2.6.20.1 preempt mod_unload 486'
should be '2.6.20-486-voyage preempt mod_unload 486'

Perhaps because I used the Debian Voyage kernel instead of the Voyage kernel sources? :( And after all the trouble I went through in finding a corresponding kernel source and building the tree and the toolkit...

Maybe there's a way to just rename just the kernel name, since everything else seems to match ... stay tuned.

The insmod / modprobe hurdle

Today, I hit a brick wall with insmod/modprobe in installing/setting up AODV on the Linux kernel.

I was able to successfully compile the module, including the kernel object files. It appeared that the modpost compilation/linking worked perfectly as well.

However, a problem came up when running the start.sh script provided - it said that it could not find the "kaodv" module, which had been compiled in the current directory.

Perhaps it had not been loaded into memory properly. However, insmod /modprobe failed to find and load the module as well.

modpost can now compile kernel modules

After my last post (which took quite a while to build all modules), modpost finally works and is able to compile kernel modules! Screenshot attached...

Long "voyage": modpost to work and compile

Ah... after weeks of hacking through kernel code and Makefiles, I am finally able to compile and run modpost , for compiling my kernel modules on Voyage Linux 0.4.

This is the story: I need to install some additional kernel modules for the WORKIT project on Voyage. All of the kernel modules pass step 1 of the Makefile, and they compile, but then I get the error:

Building modules, stage 2.
MODPOST 1 modules
/bin/sh: scripts/mod/modpost: No such file or directory

A long search for modpost lead me to find that modpost is used for compiling modules, and is part of the kernel development tools (sorry, missing the link for that post, and couldn't find it again on Google - tells you how exotic this modpost is.)

After a lot more trial and error - which included downloading and trying to recompile the Voyage Linux kernels - I found two posts today that helped immensely in coming to the last step:

• No modpost directory - Linux Forums : helped me realize how to build the modpost (but it was referencing the wrong directory; on my Voyage Linux, the code is at /lib/modules/2.6.20-486-voyage/build)

But I got the exact same error message as in the forum post: unrecognized command line option "-m".

I copied the makefile command, removed the -m directive, and tried compiling again and got a "missing elfconfig.h" error message and other errors dependent on that.

A search for "elfconfig.h" led me to a post of the Linux kernel mailing list about missing elfconfig.h which hints at running "make modules" and "make scripts".

Running those two commands on the /lib/modules/2.6.20-486-voyage/build directory solves the problem - it creates the elfconfig.h (!) file and compiles modpost. (Expect this to take a while - it makes all modules and scripts.)

CCNC BonAHA paper + iptables save

Sorry, been missing a week since I was finishing up my BonAHA paper for CCNC 2009 ... along with finalizing the BonAHA SourceForge website to be ready for the paper submission.

I also discovered that for some reason, my iptables configuration outlined in And Linux is now a wireless AP router doesn't have each time I shutdown or reboot ... which means that I have to reenter it each time.

The usual way I find this out is by trying to connect a laptop to the AP to test it ... only to find that I am not able to access the Internet through it (I get an IP address fine, meaning DHCP works.)

And running "iptables-save" doesn't work either ... so apparently, I have to find another way to save the iptables configuration for the box.

Need ncurses, module-assistant to build Linux kernel?

I think I may be getting somewhere with my compile problems for building Linux kernel modules for Voyage.

First of all, modpost apparently comes with linux-headers and is built when the kernel source is run with a "make" command.

Secondly, building the kernel apparently requires ncurses, which was causing most of my module build scripts to fail.

Several posts are recommending moving from modpost to module-assistant for Debian (and here) ... but alas, most modules I am working with are already configured to work with modpost. :( But something to keep in mind if I am writing my own kernel modules.

UPDATE, June 25, 2008 3:45 pm: I installed ncurses using this command:

apt-get install libncurses-dev

and I could indeed build the kernel without any problems.

And Linux is now a wireless AP router

I followed the instructions in the YoLinux article on making the Linux gateway ... and it works perfectly. The WORKIT AP now functions as a gateway router, instead of the bridge as it was earlier. (The LinuxJournal article mentioned in my previous post is a little old at this point.)

I just had to follow the instructions to set up the gateway. This is what I used for the WORKIT AP (eth0 is my wired WAN connection and ath0 is the wireless LAN):

   # Delete and flush. Default table is "filter". Others like "nat" must be explicitly stated.
  # Flush all the rules in filter and nat tables
  iptables --flush           
  iptables --table nat --flush
  # Delete all chains that are not in default filter and nat table
  iptables --delete-chain    
  iptables --table nat --delete-chain

  # Set up IP FORWARDing and Masquerading
  iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
  iptables --append FORWARD --in-interface ath0 -j ACCEPT

  # Enables packet forwarding by kernel
  echo 1 > /proc/sys/net/ipv4/ip_forward             

Also, while instructions exist for both iptables and ipchains, you will find yourself using the iptables rules since ipchains was RIP after Linux 2.4 came out.

Setting up a Linux AP: Bridge vs. router

Now that I have DHCP and DNS set up, the last thing I need to do is set up the WORKIT box to act as a complete AP by bridging the wireless (LAN) and wired (WAN) connections.

I naively followed linux.com's instructions on setting up a wireless AP by bridging the connections.

I was surprised to see that it completely broke the DHCP/DNS/gateway system I had set up - my wireless client instead picked up the IP address (DNS and DHCP) from the department's regular network. And my "gateway" (the WORKIT AP) had "disappeared", since I was not being assigned an IP address from it and was not on its internal network (10.0.0.x) that I had set up.

What had happened? I finally realized I had set up Linux on the WORKIT AP as a glorified wireless bridge (think switch/hub), not a router!

What I now need to do is set up Linux to act as a gateway, so the WORKIT AP acts completely as a full AP.

A little more search leads to YoLinux's article on setting up a Linux gateway and on Linux Journal as well. Basically, this involves a little fiddling around with iptables...

Installing Kernel AODV for Linux

I attempted to install the "Kernel AODV" kernel module today for the WORKIT AP. Progress was relatively smooth till the last step of the make process, which called "modpost" to create the kernel module.

As expected, there were some issues with my header include files, so had to change the Makefile and include the following lines:

CFLAGS += -I/usr/include/
CFLAGS += -I/usr/include/linux/
CFLAGS += -I/home/linux-2.6.20.1/include

(the last one is the custom directory where I untarred the kernel source.)

In addition, I also had to copy the following files from the kernel source directory:

• irq_regs.h
• device.h
• pda.h

to /usr/include/asm, since the compiler was complaining that they were missing.

After moving all the files, the compilation went smoothly and produced all the .o files; however, it is now stuck at the last step of actually making it a kernel module using "modpost". More information coming soon.

Installing kernel source in Voyage Linux

While making kernel based modules, I got an error of the type:

"Missing .config file in /lib/modules/2.6.20-486-voyage/build"

The reason this happens is because the kernel source is not installed with Voyage Linux, but if you are building kernel modules, you need access to this.

apt-get install kernel-sources does not work either.

What I did was:

1. Download the corresponding kernel source version from the Linux kernel archives.
2. Untar to /lib/modules/2.6.20-486-voyage/build
3. Run "make config"
   
4. Make sure to turn on "kernel support for .config files" (turned off by default, this is one of the first few options)

Now, you have a fully functioning kernel source tree in your Voyage Linux installation so you can compile kernel modules.

UPDATE, June 25, 2008: From a forum post, I found that you can actually find the source code for the Voyage Linux 0.4 kernel from their website at : http://www.voyage.hk/dists/0.4/linux/.

Actually, it looks like most of their distros have source online as well. Just replace "0.4" in the above link with your distribution number and expect to find your source tarball there.

Resolving compile problems with PSU's MAR - include directories and postgresql-dev

Today I attempted to install PSU's Multiple Access Router component for WORKIT. However, I found that I had to set up the include directories in the makefile in order to get the component to compile.

These are the lines I added to the CONFIGURE that is used by the Makefile to get it to work:

CFLAGS += -I/usr/lib/gcc/i486-linux-gnu/4.1.2/include/
CFLAGS += -I/usr/include/c++/4.1.2/
CFLAGS += -I/usr/include/c++/4.1.2/i486-linux-gnu/
CFLAGS += -I/usr/include/linux/

In addition, I also had to install the Postgresql-dev package since it complained of "libpq-fe.h" missing. I installed it using:

apt-get install postgresql-dev

I did have to change one setting that was causing a conflict with the INCLUDE directories for the root Makefile. This file was /dotconf/Makefile, and I commented the following line:

# CFLAGS =

dnsmasq now working as DHCP server as well

Now I have dnsmasq working as a DHCP server as well.

In my configuration, I added the following lines:

dhcp-range=ath0,10.0.0.10,10.0.0.250,24h
dhcp-leasefile=/var/tmp/dnsmasq.leases
dhcp-authoritative
# Gateway
dhcp-option=3,10.0.0.1
# DNS
dhcp-option=6,10.0.0.1

and dnsmasq is now able to serve up DHCP addresses on the fly to clients.

dnsmasq now works for DNS requests - after initial "query refused"

For a long time, I was having a problem where dnsmasq refused to respond to DNS requests and gave an error message "query refused".

At first I thought that it might have been because dnsmasq wasn't running or not binding to the proper port, but after using "netstat -ap" and running "dnsmasq -d", it was obvious that dnsmasq was running and able to receive the queries.

So why wasn't dnsmasq failing to do IP lookups and interact with the upstream DNS servers? Even though from most documentation (such as on Debian and ubuntu.wordpress.com) suggests that a simple apt-get would do the trick?

Anyway, I found that my /etc/resolv.conf had only 127.0.0.1 listed as its nameserver, and dnsmasq was using this as its resolv.conf . I looked around to see if I could find another file that the resolvconf binary was generating that had the actual nameservers, and I found this file: /etc/resolvconf/run/interface/eth0

I change a line in /etc/dnsmasq.conf to read:

resolv-file=/etc/resolvconf/run/interface/eth0

And hey presto, it worked!

Network (eth0) automatically gets IP address using DHCP now

I recently ran into a strange problem with the Voyage/Soekris box, where the box would not get the IP address using DHCP on bootup, even with the network cable plugged in. However, when I manually reset the networking (using ifdown/ifup eth0 or /etc/init.d/networking restart), it would work properly.

This was strange, as it had been working fine all along, and I had the /etc/network/interfaces set up as follows:

auto eth0
iface eth0 inet dhcp

After prolonged searching on the Internet, I found two entries on Ubuntu forums:

• No more automatic dhclient on bootup
• Network not connecting automatically on boot

that helped, but didn't quite work in my case.

I finally added a line to the /etc/network/interfaces file to make it read like this:

auto eth0
iface eth0 inet dhcp
pre-up ifconfig eth0 up

and then, hey presto, the network card now picks up an IP address automatically on boot.

Fixing dnsmasq problems on Soekris/Voyage Linux

(Sorry, my research blogging has been intermittent - but it does affect my documentation, so I am trying to be consistent about it.)

This morning, I needed to look at the DNS and DHCP servers on the Soekris box - somehow, it wasn't working perfectly.

The problem started when I first tried to install dhcpd (the DHCP server) last year without realizing dnsmasq had already been installed - and without knowing what dnsmasq is. After some frustrating error messages and doing some Google searches for dnsmasq and dhcpd, I finally found out more about how they work.

This article gives a good overview of how to set up dnsmasq.

Updates: LightPeers, WORKIT, Spring semester

A few updates since my last blog post, and this is a quick summary:

1. I finally discovered a related work to BonAHA called LightPeers. LightPeers appears to be a PhD dissertation done in Denmark, and the aim of the project is the same: develop a robust application framework for mobile P2P or ad-hoc applications. There are a few differences between BonAHA and LightPeers, but the basic concept is the same.
2. I got the WORKIT MIPClient finally working and able to connect to a wireless AP! Yay! Now to get it connected to the WORKIT AP and test its connectivity to that system.
3. Almost at the end of the Spring semester now. That's good and bad ... because I'll be tied up for the next two weeks with project demos, TA grading and reports ...
   

Photos of Soekris net5501 in Demarctech ODE-8585 Enclosure

At long last, I have photos of the Soekris net5501 in a Demarctech ODE-8585 enclosure.

This Soekris board is currently running Voyage Linux.

Thanks go to George at Columbia Univ's Physics Machine shop for putting this together.

Ad hoc applications

For our 7DS project, we are dealing with a class of applications that are highly mobile, work in disconnected or intermittently connected networks, and without central servers such as DNS and DHCP.

What name do we give this new set of applications? We call them ad-hoc applications!

The only reference I was able to find for ad-hoc applications seems to suggest that this term isn't very popular, but maybe it will catch on...

Our 7DS system - and the BonAHA framework I am building at Columbia and has been put on SourceForge - will hopefully create a usable framework for these class of ad-hoc applications. We're nearly done with version 1.0, but I still need to fix up some bugs and document the code.

Drill Bushings from McMaster-Carr for WORKIT Soekris

Hi, I'm back after a long while ... decided to continue using my Blogspot account for work related blogs, while the CS@CU wiki can remain as a wiki for official documentation.

Today, I was looking at finding ways of getting washers/bushings to seal off the Demarctech hardware that encloses our Soekris net5501 box. Talking to George at the machine shop indicates that McMaster-Carr would be the best source of such bushings.

I think I am looking for the items under :
Machining and Clamping > Drill Jig Bushings > Drill Bushings and Liners > Drill Bushings for Soft Materials

More to come soon ... as soon as I figure out exactly what I need.