Slice with a Pinch of Salt?

Go is unlike any other language. Often programmers have a muscle-memory of doing certain things in a certain way in their native language — like Python, Ruby, Javascript, etc. — and they try to re-apply the same in Go, and end up feeling skeptical about the whole idea. Well, Go is different! It is swift like others, but is also capable as a system programming language.

One such case is the Slice [1] specification in Go.

Most languages, like Python, create another copy of the underlying array when any of the slices pointing to it does a write. This is classic Copy-on-Write (CoW) operation.

Go takes a more lean and lazy approach in doing this. It keeps modifying the same underlying array until the capacity of a slice is reached. What do we call this, Copy-on-Capacity-Overload (CoCO)?

Yes, this could be a weird semantic, and seems to be implementation specific. Strangely, the spec only defines limited behavior (as below), and does not say much on how multiple slices may or may not refer to same underlying array depending on when the append hits capacity. More on this after the examples.

If the capacity of s is not large enough to fit the additional values, append allocates a new, sufficiently large underlying array that fits both the existing slice elements and the additional values. Otherwise, append re-uses the underlying array. [1]

Continue reading “Slice with a Pinch of Salt?”

Golang: Handy Stack Trace

Go can feel indispensable and tempting at times, and it has only got some makeshift-debuggers, here and there — nothing full-fledged. If you try debugging through tens or hundreds of goroutines, it becomes even more confounding and hairy!

A little bit of runtime, a little bit of reflection, that’s all it is.
Continue reading “Golang: Handy Stack Trace”

IPv6: Floating IPs and Duplicate Address Detection

The very nature of the floating IPs can lead to some classical quirks in a distributed system network. This discussion mainly focuses on IPv6, and how its duplicate IP detection mechanism can clash with the floating IP technique.

Floating IPs are a common scenario in Highly Available or Scaled-out Distributed Systems. The basic idea behind it is to have a transient IP address that can move from one node to another, keeping the change of serving-node transparent on the access-side of the network. For instance, if there are two server machines, each represented by an unique IP, and one of them goes down, then its IP address “floats” to the other server which will henceforth process the client requests. This technique is widely used to provide seamless transition from one serving-node to another in case of failures. One such implementation is present in OpenStack Nova.

On the other hand, Duplicate Address Detection (DAD) is a mechanism to identify if same IP is assigned to multiple nodes in a local network. It is implemented using the Neighbor Discovery Protocol (NDP) under IPv6. It uses the Neighbor Solicitation (NS) and Neighbor Advertisement (NA) messages. The operation is applicable to all the IPs that are link-local. More specifically:

  • all the IPs that fall under the link-local address-family
  • all the IPs that fall under global address-family but are present on the same LAN (one hop away on the link)

Continue reading “IPv6: Floating IPs and Duplicate Address Detection”

Linux and Application Packaging

This is more of a critique than a commentary!

Getting an older version of a software source-code to compile, link and run on a new Linux system is excruciating. Those who have dealt, know better!

Any Linux binary should be runnable on all Linux distros of any version. This is “the requirement”! This is not currently possible, and we live with it. But it balks us when a product has system version upgrades, and we end up managing parallel lines of versions of libraries, compilers and linkers — just to make the old stuff run. Continue reading “Linux and Application Packaging”

Debugging Linux Kernel using QEMU

An old lost nifty toolkit for kernel debugging, I used long ago… Of course, the host OS is Linux — some Debian clone (Ubuntu).

Commands to run qemu and debug Linux kernel (either one will do):

qemu -kernel bzImage -append "root=/dev/sda console=ttyS0" -m 2G -hda wheezy.img -serial stdio -nographic -nodefaults -s -S
qemu -kernel bzImage -append "root=/dev/sda console=tty0" -m 2G -hda wheezy.img -s -S

Two important options here are:

# -S  Do not start CPU at startup (you must type 'continue' on the gdb prompt).
# -s  Shorthand for -gdb tcp::1234, i.e. open a gdbserver on TCP port 1234.

On the debugging side:

$ gdb bzImage
(gdb) target remote localhost:1234

Continue reading “Debugging Linux Kernel using QEMU”