Alex on Linux» Opinion

Python for bash replacement

Alexander Sandler — Sun, 26 Dec 2010 16:25:37 +0000

When I started learning Python, I was looking for a programming language that would replace BASH, AWK and SED. I am a C/C++ programmer and as such I better invest my time into studying C and C++. Instead, every time I needed some complex script I opened up a book on BASH and refreshed my knowledge. And since bumping into boundaries of what BASH can do is relatively easy, I always opened awk/sed book few minutes later.

Actually, this is quiet common. Once in a while I see my colleagues, just like myself, open up a book on BASH. The problem is that because we don’t actively program BASH, the knowledge and experience that we gain from this experience wear out over time. So next time we approach, so we have to repeatedly study BASH stuff over and over again. And again, this is not only BASH I am talking about, but also AWK and SED.

It is utterly broken state of affairs and I wish there was a solution. Unfortunately there is no solution yet. The good thing is that with some effort the solution may arise. I am talking about Python programming language.

Needless to say that Python can do everything that BASH can do. However, it was never designed as shell environment and as such it is hardly convenient.

For instance, running external commands is easy. You should use subprocess module to run the command and there you go, you have both return value and the output. But this is miles away from convenience of simply typing the command and hitting enter.

Another issue is input/output redirections and piping. Regular shell does this with ease. Python obviously does this as well, but not without a hassle of importing the right module and creating the right object, etc.

On the other hand things are not that bad. For instance implementing command line should be somewhat easy in Python because it has native support for completion (which it uses in its own command line interface).

There is a project that makes few steps in the right direction. I am talking about IPython of course. Unfortunately, I doubt that IPython will ever be able to replace BASH. It was never their goal. Even though IPython developers implemented some features that make IPython command line a little down to earth, I am in doubt they will continue moving in this direction. Also, I must say that IPython evolves quiet slowly.

What do you think? Will Python ever be able to replace BASH?

My next programming language

Alexander Sandler — Sun, 22 Aug 2010 21:04:43 +0000

This week-end I’ve been playing with various version control systems. Until now, I’ve been doing all my home codings with subversion. I’ve written about bazaar in the past, but it seems to me that bazaar isn’t going anywhere and it busts any piece of motivation that I have to continue writing about it.

Version control that I did try is git. This is a very popular version control system and for a good reason. Comparing git and subversion brought me to a conclusion that git is really a very consequence of how things work in the world. Thing about git is that it is a distributed version control while subversion is not. You can make a distributed version of subversion, but it won’t be subversion anymore.

Distributivity is one thing, but there are more. Take the standard trunk/branches/tags layout that you have to create in subversion – version control could do it for you, as git/bazaar/mercurial do. At first, after working with CVS for some time, having an option to have non-standard layout seemed cool for some time. But then it appeared completely useless.

This brings a notion of evolution into version control systems and I bet there’s similar process with programming languages. I think we can already starting drawing a programming language that will replace Python.

Yep, Python isn’t perfect. I thought I’d compile a list of things that shall be different in ought to be programming language that comes after Python.

Passing self to methods is counter-productive. Using self to access members is super counter productive. Yet this is how Python works. Without this, it won’t be Python anymore.
Same thing with global. Every program has this one global variable that every other class and function use. So, every entity that uses that variable has to declare it with global keyword. Utterly counter-productive.
Starting every private method and field with __ (double under-score) is ugly. The way it changes method name prepending it with a class name is even uglier. It is a really hackish way to handle this problem. I understand that this is the Python way of doing things, but it feels wrong.
Another hackish thing are decorators. Make an option to have static methods, class methods and properties and decorators are gone. No one would need them anymore, and it will be one bit easier to master Python.
You name it.

Disclaimer (can’t get along without one this time ): This post may seem like one criticizing Python and it does in some ways, but I doubt these can be fixed in something called Python (unless Guido van Rossum brews it and calls it Python 4). In the meantime I love Python and use it on a daily basis.

Distributed vs. centralized version control systems

Alexander Sandler — Tue, 16 Jun 2009 19:26:33 +0000

When git appeared for the first time, I was convinced that I don’t need it. It seemed that git is good for large and distributed projects, like kernel. While my own needs where much more modest – manage a project with couple of dozen files and perhaps a couple of contributors at most.

Couple of years passed and I am now looking into Bazaar. This is a relatively new distributed version control system. I think I’ll write more about Bazaar, but in the meantime, I’d like to share some thoughts that crossed my mind while I was playing with it.

There’s one thing that distributed version control does much better then centralized version control. Let me explain.

We do commits for two reasons.

We want our work backed-up. We commit to repository because we know that repositories being backed up.
We want to be able to track our work, i.e. keep a record of changes that we apply to the project.

In centralized version control system, these two are integrated in one system. I.e. if you want to track your changes, you commit. If you want to backup your code, again, you commit.

Don’t know how about you, but for me, this is totally broken. I work on a small projects, usually from home. I prefer not to host Subversion server on one of my home computers, but rather have a small VPS system for this purpose. It has mirrored disks, so the data is safe, more or less.

The problem is that this setup is horribly slow. It uses web front-end, so it creates a new secure connection to the server for every file I commit. This is so frustrating. I bet Subversion has faster front-ends, but I doubt they will be fast enough.

On the other hand, with distributed version control, the two above items are separated. When you commit for code tracking purposes, you commit to your local repository. When you want to backup your code, you commit to centralized, backed-up repository. In the first case, you can commit as often as you want – do local commits, in Bazaar terms. In the later case, you can commit to the repository, lets say, nightly.

So, despite distributed version control systems naturally fit large projects, my mini-projects can enjoy them as well. Hence I am definitely going to try Bazaar with one of my mini-projects and report the results. Stay tuned.

6 things that I miss in bash

Alexander Sandler — Tue, 09 Jun 2009 18:20:36 +0000

What is the most frequently used piece of software on your computer? Here’s one program that holds one of the highest place in my list of most frequently used programs. I am talking about… bash.

In case you don’t know, this is the program behind large portion of the Linux command line. It is the program that actually makes majority of work turning Linux command line into such a magnificent tool it is.

As for use frequency, on my computer the only program that beats it is perhaps a web browser. I use it more often than mail client, text processor or just any other program.

Now, one thing that entertains me is that there is absolutely no hype around bash. I mean, you probably know that next version of Firefox coming tomorrow (I am writing this on June 9th, 2009). But do you know when next version of bash is coming? Do you even know what is the version of bash that you currently use? Well, you are not alone – I have no clue either

One thing that concerns me is that bash almost didn’t change for the last ten years. I mean the only new feature that, to my knowledge, it didn’t support ten years ago is bash_completion – that is the advanced version of plain completion that it does to files. I could be wrong about this of course, but one thing is certain. Whatever the changes are, they are unnoticeable.

So, I thought that I’d compile a list of features that I would like to see in the upcoming version of bash. Now really, if only one of them would become a reality, I’d chew my pair of orange Crocs (I do have a pair of orange Crocs), put it on video and post it on youtube. Alright, I won’t do that. But it would still be very nice to see these happening.

The list:

1 – History sharing

I often open multiple shells. It would be really nice to be able to synchronize history between them. Right now, it seems that bash writes history file when it shuts down. If you open another session after history has been saved, you can access commands that you’ve typed in in the previous session. Otherwise, the history between two sessions is completely independent. Not good.

2 – Completion of directory and file names based on their usage

I think that 90% of cd‘s that I do are all destined to the same directory. Why on earth I see the list of possible destination for cd in alphabetical order every time I press TAB?

3 – Completion of make targets

I know this has already been implemented. But I am wondering why it isn’t part of bash_completion package on Ubuntu. OpenSuSE is no better – it does completion only to some yast stuff.

4 – Show a browseable and searchable list of files when hitting TAB key twice

I guess you all know that. Single TAB key completes a word if it can make a single match. Double TAB shows a list of choices for completion. In case the list is too long, it builds a long list and pipes it through more.

What I would like to see is a browseable and searchable list. One that I can browse with arrow keys. I also would like to be able to enter portion of the file name and in that case bash should filter out everything that doesn’t match the criteria I’ve entered.

5 – Completion of scp, on remote host

I guess it happened to all of us. Copying file to a remote computer with scp, is always difficult because you actually have to remember the path on remote computer. Wouldn’t it be great if bash could complete remote paths as well?

Now, I know this is very difficult to implement. Basically you have to spawn ssh with ls command, wait for it to complete and parse the results. Oh and there is a password thing. You gotta take care of password prompt that ssh may produce. Have no idea how to solve this issue. I do see a couple of options, but all of them are far from being ideal.

6 – Fix typos

This one is simple. As long as I don’t hit the TAB key, no need to fix anything. Once I hit the TAB key, shell should check if the command I’ve entered makes sense and if not try to fix it. Good example of what could make sense are file names. vi probably means that you want to edit a file. If the file does not exist and yet there’s another file with almost identical name, fix it. Same with cd. You don’t cd into directories that don’t exist. So once shell detects an attempt to do so and the user asks for help (by pressing the TAB key), shell can fix the typo and make the user smile.

This is it. I hope to see any of these happening one day. In the meantime, enjoy the list and by all means leave comments with your ideas of bash enhancements.

Is desktop Linux too fragmented to succeed?

Alexander Sandler — Tue, 12 May 2009 08:07:39 +0000

This question is especially relevant after yesterday’s fiasco. I ran into an article whose name is exactly Is desktop Linux too fragmented to succeed?

The article argues that the fragmentation is what keeping Linux Desktop from beating Mac OS and even Windows. It is because the effort to create Linux desktop is scattered across multiple distribution, it cannot become a serious player in desktop solutions.

Toward its end, the article gives optimistic prospects. There are some efforts to bring Linux desktop community to common ground, unanswered though.

This is so wrong

The truth is that are some points in the article that made my eyebrows go up. You see, having single desktop Linux distribution is impossible. This is against the nature of Linux and how its being distributed.

To get it moving you have to talk big distributors into this. But for big distributors, their desktop solution is one way or another their selling point. RedHat compete with Novell through Fedora and OpenSuSE. Moreover, they don’t want common ground. They want to compete.

Small distributions want their piece of the big cake too. Asking them to provide the same desktop solution would be as asking them to close their business – if they wanted to be a part of joined effort to make the world better place and Linux better desktop OS, people behind the distro could simply join someone else’s project.

Actually, this entire idea seems to be taken from socialism. We take something from everyone and create something that everyone would use. Sounds great, but this creates so called moral hazards. This strips away the will of competition. And competition is the driving force behind many of the technological marvels that we see today in general and in Linux in particular.

The only way to improve Linux’s position in desktop market is

to invest money into development of better Linux desktop.

Actually the author of the article was addressing the same issue, but from different angle. Linux desktop needs money. It needs more graphical designers, more QA engineers and more software engineers.

It straggles to become a serious desktop operating system because no one did usability testing. Because no one hired best UI designers, as Apple did. Because no one had hired enough QA engineers to make sure that all the software indeed works.

One way to overcome this problem is indeed by combining the effort. The problem is that Fedora isn’t there to beat Windows in desktop OS market. Fedora is a beta version of RedHat’s server systems. Same can be said about OpenSuSE, Ubuntu and most of the Linux distros.

Obviously money can be spent on nothing useful. Hundred engineers will not do in one month what ten engineers do in ten month. However, financing creates opportunity. And opportunity is exactly what Linux desktop needs.

Ubuntu 9.04 with Wubi – Failure

Alexander Sandler — Sun, 10 May 2009 20:03:20 +0000

Once every year or so, I get so frustrated with Windows Desktop that I decide to install Linux. I am a big fun of Ubuntu Linux. I use it for many things, this includes a server platform for this web-site. So most natural choice for me was to try Ubuntu 9.04, the latest version. I installed it on my Laptop computer using a program called Wubi.

Wubi is a program that installs Ubuntu Linux on top of installed Windows. I already had a Windows installation on my laptop, that I didn’t want to destroy. This is why I’ve chosen Wubi. It places root file-system in a file on Windows partition. Because of this feature, you don’t even need a partition to install Ubuntu.

What got me into this

What got me into Linux desktop frenzy this time, is Vista’s slowness. We all know various virtues of Microsoft’s operating systems. But installing Linux desktop instead… Silly me. It took me just four hours to figure out that I am better with slowness. Four hours I usually don’t have.

I am full of respect to developers of Linux. I work with Linux and develop for it for many years and I think this is a great server platform, unmatched by any other OS. But, Linux desktop is unfortunately isn’t too mature to be a serious match for Windows as a Desktop OS.

Knowing this from my previous attempts to install Linux desktop, I tried a less destructive option out of all. That is, Wubi doesn’t change your existing Windows installation. It install Ubuntu Linux as if it was another Windows program.

Installing Wubi

Problems started with Wubi itself. I live in Israel and although I use English version of Vista, Wubi installation has decided that I am better with Hebrew. Entire installation procedure was in Hebrew.

Next, despite I explicitly told Wubi that I want English version of Ubuntu, it installed Hebrew version. I checked on Wubi’s web-site and it appeared someone reported this particular bug just few days before. And it already had a solution – developers presented a new revision of Wubi which wasn’t yet publicly available, but solved the problem. The installation interface language remained Hebrew, but at least it installed English version of Ubuntu.

Now in case you wonder why I had to reinstall entire operating system just because of interface language, I’ll explain you. You see, I am not very familiar with Ubuntu’s graphical interface and I didn’t know how to configure wireless network. Hebrew interface made it even more complicated for me, so I decided to get things right form the start and not to begin tackling problems on unfamiliar ground.

On the other hand, because I didn’t have internet access, I could not just install English support it as I would do otherwise.

Now to Ubuntu

My next task, after installing the Linux itself, was to configure wireless network. The installation program did detect both wired and wireless card. However, the network interfaces configuration program didn’t show the wireless interface – only the wired. Luckily, simply pressing Add solved the problem. New dialog, popped up, asking me what is my SSID.

I am lucky to have some background in wireless networking, otherwise I would not know what SSID is. In case you’re wondering, this is the name of your wireless network. Anyway, it appeared that I’ve forgotten name of my wireless network and I had to use my cell. phone (which luckily supports wi-fi) to find the name. Not to mention I’ve absolutely forgotten that I’ve used WPA2-PSK security for my network. I had to choose it manually too, but before that I had to investigate it with my cell. phone.

I must say that Ubuntu could ship a program that scans for wireless networks, but it just wasn’t there.

On the bright side, once I entered right values for SSID and encryption, it got connected immediately.

The breaking point

The next thing I planned to do was mounting my Windows partitions. Somehow mounting my C: drive wasn’t that difficult. The desktop was empty. I right clicked on it, created a new directory and then double clicked on it. It opened gnome file explorer. Then I pressed the computer button and here they were, my CD-ROM and C: drive. Clicking on C: drive, created a C: drive icon on the desktop and I could safely delete the directory I created, knowing that I can always open gnome explorer by clicking on C: drive icon.

Now I know that there are several ways of opening gnome explorer, but I just couldn’t find them.

Anyway, note that I have two partitions and gnome explorer only showed me one of them. It appeared to me that something is wrong with D: drive and I should try to mount it. I went to Add/Remove programs and added a program called Disk Management. It seemed logical to me that I need some program that manipulates disks, so I searched for disk and found that Disk Management program. However, once I started it, it reported that I don’t have disks at all.

This was the breaking point. Playing around with Ubuntu was fun, but I had to stop and go back to the world of productivity – where at least sometimes things work.

Conclusion

Linux is a great operating system and like I mentioned I feel lots of respect to developers of Ubuntu and Linux for the job they are doing. This is especially true because I owe my income to Linux, for the last decade. Linux has given me a profession and I am grateful for that. However it seems that, regrettably, Ubuntu, one of the the undoubtful leaders among Linux distributions, is still too immature as a Desktop solution. Too many things are not obvious. Too many things just don’t work out of the box. Yes I could find a solution for everything, but I am just too busy person to search solutions for everything.