Unfortunately, various sources of information on UML tend to over-complicate things. I am not software architect and drawing UMLs is not my job. So my UML skills are poor by definition. Moreover, I am happy with this situation and don’t see it changing in the future (even if I get promoted ).

So from time to time I need a simple UML reference card. Simple search finds references like this one, which are excellent if you are serious about UML, and I am not.

Eventually, I decided to write a short UML class diagram reference card for myself. I hope you will enjoy it as well.

Inheritance

So, this is how classes inherit one from another. Here Child class inherits from Parent Class.

Use

This is how User class uses Resource class.

Contains and manages

Here, Whole class contains and manages Part class. This type of relation can be extended to one of the following:

• One to One
• One to Many
• Many to One
• Many to Many

References

Here, Whole class references to Part class, but does not manage it. Again, can be extended with:

• One to One
• One to Many
• Many to One
• Many to Many

This is enough information for now. I’ll probably extend it over time. In any case, please post your corrections and suggestions.

No related posts.

In case you’re wondering, when write(), fwrite() or any other library call that writes data to disk reports success you are not guaranteed that the data is actually on the disk. In fact, in Linux, write() reports success when data is in dirty cache. Then, special kernel thread kicks in and makes sure that the data is on disk.

Depending on circumstances, it may take some time until writer kernel thread will finish writing. Anyway, in his post Evan talks about how to make sure that the data is actually stable on disk.

Related posts:

1. What is direct I/O anyway?
2. Multithreaded simple data type access and atomic variables
3. Models for multithreaded applications

Lets say we have a multi-threaded program.  The program should handle heterogeneous tasks. On a very basic level, our program probably receives some data, processes it and produces an output. Question of how to manage threads usually arises in performance critical applications. Otherwise, it is quiet simple – we squeeze into single thread as many tasks as possible.

Actual data processing may take several steps. We may want to split it to several tasks, each handled by thread of its own.

Thread per task model

When using this model, we allocate single thread per task. Since we want to pass work from one thread to another, we will need some synchronization mechanism.

I suppose we’ll need some queues to pass things between threads. Also, we may need muteces or semaphores to synchronize access to queues and other shared objects.

One obvious weakness of this approach is performance. When moving task objects from one thread to another we often move them from CPU to CPU. Modern CPUs heavily deploy caching mechanisms. So keeping objects local to one CPU brings significant performance boost. By moving task from one CPU to another we heart caching and cause performance degradation.

Another performance related problem is that while your task object moves from one thread to another it should undergo at least one context switch. Depending on overall number of tasks, time it takes to process single task can be greater. Another cause of context switches is mutual exclusion mechanism that have to protect the data structure used to communicate objects between threads.

One last significant performance issue that I would like to mention is in the core of this approach. Each thread is bound by processing power of a single core. But what if one of the tasks require more processing power. The computer may have enough processing power. This is especially true with recent multi-core trend. But when a single thread runs on one core at a time inevitably each thread processing the request, is a potential bottle-neck.

Advantage of this approach is in its simplicity. You don’t have to develop significant infrastructure to make it work. All you need is pthreads and a couple of classes that do the job. All tools that you will use are available out of the box. Because it is so common, it will be easier to find skilled and experienced engineers to implement and use with this model.

Worker threads

In this model, each task split into multiple jobs. Each job being handled by single thread. There is number of worker threads each taking care of at most single job at any particular moment. To improve performance, system tries to handle jobs that belong to same task on the same core. This means that there might be number of threads per single core.

Unlike thread per task model, this model is much more difficult to implement. It implies some constraints on the tasks:

1. Engineer designing such system should be able to split tasks into multiple jobs. Alternatively, tasks can be very short.
2. Jobs should be more or less equal in size in terms of required processing resources.

In addition, this approach requires smart task management. Such system should implement scheduler that spreads tasks between cores. A serios problem arises when number of running tasks becomes larger than number of cores in the system. When it happens, scheduler should start two or more tasks on single core. To do it wisely, it should be aware of how much processing power required by tasks. It should manage number of tasks running on particular core and assign new task to least busy core.

Needless to say that this approach is way more complex than naive thread per task model. It requires more skilled engineers. Moreover, these engineers should spend their time developing something that is not directly related to the matters of the project – that is various infrastructures for the project.

On the other hand, this approach allows engineers to squeeze every bit of performance out of the machine. This approach will take the hardware to its limit.

Other approaches

There are many other approaches of course. Most of them however, are an attempt to merge between two models I described. For instance, one company started with thread per task model for their embedded system and quickly ran into performance problems. So they tried to solve problems by splitting busiest tasks into multiple threads, basically changing threading model for one particular thread into worker threads model.

Recently I ran into another model that is somewhat unique. This model implements so called user-mode threads. Entire program runs in a single thread. Like worker threads model, in this model tasks split into smaller jobs. However, instead of letting the operating system to schedule between job execution, this model implements a user-mode scheduler.

The actual scheduling is made using setjmp() and longjmp() calls. One obvious constrain that this model presents is that if thread goes to sleep in operating system terms it will send entire system to sleep. As a result, the system should have very tight control over what threads does and in particular how it sleeps. I.e. threads are not allowed to do regular I/O. Instead they should do I/O in a centralized way.

For example, thread that wishes to read from a file-descriptor should register this file-descriptor with scheduler. Scheduler will select() or poll() on all registered file-descriptors. Depending on what file-descriptor requires attention, scheduler will schedule thread that takes care of that file-descriptor.

In this model you can run only a single instance of scheduler on single processor. So this model is not entierly optimized for MP machines. If you still want to run multiple schedulers on single computer then you can either run them in separate processes or multiple threads of the same process. In both cases communication between threads that belong to different schedulers is cumbersome. For instance if run multiple schedulers in multiple threads, you cannot use mutexes to synchronize between threads that belong to various schedulers because locking such mutex will send entire system to sleep. Instead you should use some sort of I/O based IPC.

Despite number of disadvantages that this approach has, there are couple of good things into it. Synchronizing between threads that belong to same instance of scheduler is much easier because there is, at most, one thread that runs at any particular moment. So, if, for instance, you have some simple data structure that being used by multiple threads there is no need to protect it from simultaneous access.

Related posts:

1. A new kind of virtualization
2. C/C++ reference counting with atomic variables and gcc
3. Why interrupt affinity with multiple cores is not such a good thing

Actually, this is quiet common. Once in a while I see my colleagues, just like myself, open up a book on BASH. The problem is that because we don’t actively program BASH, the knowledge and experience that we gain from this experience wear out over time. So next time we approach, so we have to repeatedly study BASH stuff over and over again. And again, this is not only BASH I am talking about, but also AWK and SED.

It is utterly broken state of affairs and I wish there was a solution. Unfortunately there is no solution yet. The good thing is that with some effort the solution may arise. I am talking about Python programming language.

Needless to say that Python can do everything that BASH can do. However, it was never designed as shell environment and as such it is hardly convenient.

For instance, running external commands is easy. You should use subprocess module to run the command and there you go, you have both return value and the output. But this is miles away from convenience of simply typing the command and hitting enter.

Another issue is input/output redirections and piping. Regular shell does this with ease. Python obviously does this as well, but not without a hassle of importing the right module and creating the right object, etc.

On the other hand things are not that bad. For instance implementing command line should be somewhat easy in Python because it has native support for completion (which it uses in its own command line interface).

There is a project that makes few steps in the right direction. I am talking about IPython of course. Unfortunately, I doubt that IPython will ever be able to replace BASH. It was never their goal. Even though IPython developers implemented some features that make IPython command line a little down to earth, I am in doubt they will continue moving in this direction. Also, I must say that IPython evolves quiet slowly.

What do you think? Will Python ever be able to replace BASH?

Related posts:

1. 6 things that I miss in bash
2. sed – the missing manual
3. My next programming language

I started a thread that looks like this:

try {
    do_something()
} catch (...) {
    std::cout << "Got unknown exception" << std::endl;
}

The do_something() routine eventually called pthread_exit().

Once I ran this piece of code, I instantly got an unknown exception notification. Long story short, here’s what I found out.

When calling pthread_exit() in C++, it has to destruct all objects that has been created on stack. This process called stack unwinding and this is exactly what happens when you throw an exception. pthread_exit() utilizes this feature of C++ to cleanup before shutting down the thread for good.

To do that pthread_exit() throws some obscure exception and catches it right before ditching the thread. This way it cleans up all objects nicely. On the other hand, catching … becomes impossible.

Related posts:

1. How to handle SIGSEGV, but also generate a core dump
2. C/C++ reference counting with atomic variables and gcc
3. Signal handling in Linux

Version control that I did try is git. This is a very popular version control system and for a good reason. Comparing git and subversion brought me to a conclusion that git is really a very consequence of how things work in the world. Thing about git is that it is a distributed version control while subversion is not. You can make a distributed version of subversion, but it won’t be subversion anymore.

Distributivity is one thing, but there are more. Take the standard trunk/branches/tags layout that you have to create in subversion – version control could do it for you, as git/bazaar/mercurial do. At first, after working with CVS for some time, having an option to have non-standard layout seemed cool for some time. But then it appeared completely useless.

This brings a notion of evolution into version control systems and I bet there’s similar process with programming languages. I think we can already starting drawing a programming language that will replace Python.

Yep, Python isn’t perfect. I thought I’d compile a list of things that shall be different in ought to be programming language that comes after Python.

1. Passing self to methods is counter-productive. Using self to access members is super counter productive. Yet this is how Python works. Without this, it won’t be Python anymore.
2. Same thing with global. Every program has this one global variable that every other class and function use. So, every entity that uses that variable has to declare it with global keyword. Utterly counter-productive.
3. Starting every private method and field with __ (double under-score) is ugly. The way it changes method name prepending it with a class name is even uglier. It is a really hackish way to handle this problem. I understand that this is the Python way of doing things, but it feels wrong.
4. Another hackish thing are decorators. Make an option to have static methods, class methods and properties and decorators are gone. No one would need them anymore, and it will be one bit easier to master Python.
5. You name it.

Disclaimer (can’t get along without one this time ): This post may seem like one criticizing Python and it does in some ways, but I doubt these can be fixed in something called Python (unless Guido van Rossum brews it and calls it Python 4). In the meantime I love Python and use it on a daily basis.

Related posts:

1. Distributed vs. centralized version control systems
2. Bazaar for subversion users, part 1 – the basics
3. Todo list

Today I’d like to talk about one nice trick that I learned few days ago.

When working with large software systems, memory management becomes an imperative. In C, you can easily allocate a large chunk of memory and allocate structure right on that buffer. This is by far more difficult in C++, because compiler has to call consturctor.

Apparently, you can, in a way, directly call object’s constructor . I.e. you can allocate an object, on specified memory region, without actually allocating this region.

This is how you do it.

char* s = new char[1024];

SomeClass* p = new (s) SomeClass;

First new operator just allocates 1024 bytes. This is good old allocation as we know it. Note the special syntax of the second new operator. It allocates the new object on memory specified in brackets. Basically, this calls SomeClass’s constructor using s as storage.

One thing that I don’t know how to do is how to call destructor on the object – i.e. how to delete an object in place.

Related posts:

1. Direct IO in Python
2. One sane voice in the crowd
3. Recursively deleting symbolic link and what it points to

I think this is going to become my new icon. What do you think?

No related posts.

At first, I thought less reads entire input first. This would make standard input stream free to process key presses from the user. So I decided to check this out. I created a 1Gb long text file and ran less on it. I expected less to take some time to show file contents, but it showed first lines of the file instantly. Also, it didn’t consume 1Gb of RAM as I expected.

The conclusion is obvious. less does not read entire input buffer before letting user to interact with itself. Then how?

Here’s what less does. It separates input file stream from user input stream. Both inputs initially come via standard input stream, so less separates between them. First it duplicates the standard input stream. This allocates a new file descriptor. Then it closes old file descriptor, freeing file descriptor 0 for use. Then it opens /dev/tty. When opening a file, Linux uses next available file descriptor. File descriptor used as standard input is 0, so when less opens /dev/tty again, file descriptor of the newly opened file has value 0.

Eventually, it ends up with the new input coming via standard input stream file descriptor (0), and old input still available via file descriptor that it duplicated in the beginning. It reads the input file from the duplicated file descriptor, and uses curses on standard input stream.

You may be wondering what is /dev/tty and what it has to do with standard input streams. This is really fascinating stuff.

As you know, in Linux, everything is a file. So is terminal. Linux uses device files to represent various system devices. /dev/tty is a file that represents terminal of the current process. When process reads from /dev/tty it becomes its input. When program writes to /dev/tty it becomes its standard output or standard error stream.

So, when less reopens /dev/tty, it actually recreates standard input. Older descriptor, one that has been duplicated, can no longer accept new input, but less still can use it to read what it has been written into it already.

Related posts:

1. Creating new application on top of SSH
2. sed – the missing manual
3. Direct IO in Python

So, I decided to leave Fabrix and as of last week I am software engineer and senior consultant at Dell IDC (Israel Developer Center).

Related posts:

1. Resume
2. I am looking for a new job
3. What it takes to be a QA engineer