features.h: Feature Test Macros

With which C standard (C89, C99, C11, etc.) you are compiling your code decides which features are available. For example, pthread-barriers are not present under C99. If you come across a scenario when the compiler fires an error saying something is not defined, even though, it seems to be present in the included header, then probably it is because some how that definition was excluded by the preprocessor because the feature is disabled. To properly enable particular feature, we needed to define appropriate macros that are present in features.h.

Continue reading “features.h: Feature Test Macros”

The Daemon Pattern

This is a common pattern seen in daemons (in C like languages).

int some_func()
{
     struct foo_s *foo = (struct foo_s *) malloc(sizeof(struct foo_s));
     ...
     while (1) {
          ...
          bar(foo);
          ...
     }

     free(foo);
     return r;
}
pacman
 Courtsey: pacman.wikia.com

The unreachable free() makes no sense. A missing free(), though unreachable, annoys a purist. Considering the scope and visibility of foo(), putting it on the heap is not the best approach. The scope is within some_func(), and the visibility is until the process exits (killed).

Continue reading “The Daemon Pattern”

The “thing” called Agile

The “Manifesto for Agile Software Development” catalogs very practical suggestions on how to engineer a software in an effective way. These are like fundamental truths, and I believe these could be extended even beyond software engineering, so much that they could be called Management Principles for Effective Teamwork. In the title of the manifesto, “Agile” is an adjective. It wasn’t called “The Agile Manifesto” or “The Agile Methodologies” where “Agile” becomes a noun. When something becomes a noun, it becomes a commodity — something that could be bought and sold. Rather, it could have been aptly called “The Agility Manifesto” for a terse colloquial version.

“Agile is not what you do. Agility is how you do things.” — Dave Thomas

Continue reading “The “thing” called Agile”

Problem Decomposition deserves more attention!

If there is one idea in Computer Science and Software Engineering that underlies every other idea, then what is it? (It’s not Algorithms!)

“All problems in computer science can be solved by another level of indirection.” — David Wheeler

If indirection implies how a problem can be solved, then decomposition implies how a solution was reached. Problem Decomposition is about understanding the solution to a problem. It is the inverse of indirection.

Problem Decomposition is breaking a big, complex problem into smaller parts that could be solved relatively independently.

Continue reading “Problem Decomposition deserves more attention!”

Spark Internals and Design Basics

Apache Spark is an open-source general-purpose cluster computing engine built around speed, ease of use, and sophisticated analytics, that is primarily used for Big Data. It came in as an alternative to cope with the complexity and tediousness in Hadoop MapReduce for running Machine Learning algorithms. Spark introduces two main abstractions: resilient distributed datasets (RDDs) and parallel operations. Spark is written in Scala Programming Language and runs on Java Virtual Machine (JVM) environment. It very well exploits the functional programming aspect of Scala for elegance and simplicity.
Continue reading “Spark Internals and Design Basics”

TCP Checksum: The Fault in the Stars

TCP Checksum could be deemed as one of the weakest non-cryptographic checksums, and yet it continues to be there, undisputed. Sometimes edge-systems even have it turned off for performance reasons, counting on the application checksums for integrity; while other systems like gateways and servers have them offloaded to the network interface card (NIC), mostly because the functionality exists. The question is, does it really serve any purpose in today’s date? It does, very little.

RFC 793:

The checksum field is the 16 bit one’s complement of the one’s complement sum of all 16-bit words in the header and text. If a segment contains an odd number of header and text octets to be checksummed, the last octet is padded on the right with zeros to form a 16-bit word for checksum purposes. The pad is not transmitted as part of the segment. While computing the checksum, the checksum field itself is replaced with zeros.

Continue reading “TCP Checksum: The Fault in the Stars”