Understanding shebang + eval combo from ancient perl script

During the daily development, we have a collection of scripts to help us automate some mundane task. Most of them are written in perl and quite often, I feel shamed to be a programmer that only know how to use those scripts without actually taking a look at their source code. However, I don’t know perl by any measure and recently, I decided to take this challenge: I quickly went through the basic aspects of the language (with the help of this nice tutorial: Learn Perl in about 2 hours 30 minutes )  and dived directly into some scripts to start reading. Then I met this daunting code chunk from the very beginning of a perl script:

#!/usr/bin/perl
eval 'exec perl5 -S $0 ${1+"$@"}'
   if 0;

So, I spent some days digging, and finally get this code chunk clears out. I’ll try my best to explain this code chunk in a newbie-friendly way (because I am one of them :)).

History

Usually, it is unnecessary for people to know that if a script should be executed by Perl, shell, or other interpreters. In other words, they can execute the script by typing the filename and the script itself will find the right interpreter to run it. This is usually done with shebang.  In perl, the common way to do so is to write


#!/usr/bin/perl

at the very first line of your perl script. However, not all system supports shebang and most likely, those systems will run your script as if it is a shell script, which, of course, will lead to the failure of execution. In this case, we need to figure out a way to tell those systems that “even if you are running the script as a shell script, please invoke perl interpreter to interpret the content of the script” and that is exactly what that daunting code does.  Now, let’s dive into this code chunk to see how it works.

Dive in

#!/usr/bin/perl
eval 'exec perl5 -S $0 ${1+"$@"}'
   if 0;

 [1]:  First line of code uses shebang and it invokes perl interpreter located under /usr/bin/, which should be enough  for systems that support shebang to know which perl interpreter should be used to run the content of script.

[2-3]:  For system that support shebang, the system already knows the content of script should be interpreted as perl. So, line 2 – 3 will be treated as perl. Since “Carriage return” is the same as “whitespace” in perl world, line 2 -3 will not get executed because of if 0. However, for system that doesn’t support shebang, the whole script is treated as shell script and thus, line 1 will be treated as shell comment and ignored. Then, the system continues to run line 2 – 3 as shell command. There is one important difference between perl and shell (i.e., bash) is that perl will spot the continuation line (because carriage return is the same as white space) but not bash. So, shell will first execute line 2. For now, let’s just say line 2 executed by shell will tell the system that “to re-run the whole script again under perl and where to find perl interpreter” (we will investigate more in detail in the following section). So now our goal is achieved: system that doesn’t support shebang will go ahead to use specified perl interpreter from line 2 to re-run the whole script as perl script. During the re-run, line 2-3 will get ignored by perl.

Line 2: what the heck is this?

Now, let’s study code from line 2 in detail.

eval 'exec perl5 -S $0 ${1+"$@"}'

The eval in shell takes a string as its argument, and evaluates it as if you’d typed that string on a command line. So, the shell actually executes

exec perl5 -S $0 ${1+"$@"}

$0 get expands to the name of the script by shell. However, ${1+”$@”} looks quite mysterious. It involves an ancient Bourne shell bug (if no argument provided, it uses an empty argument instead of nothing) and the article What does ${1+”@”} mean explains it very clear:

The ${1+"$@"} syntax first tests if $1 is set, that is, if there is an argument at all.
If so, then this expression is replaced with the whole "$@" argument list.
If not, then it collapses to nothing instead of an empty argument.

Aside note on ${1+"$@"}, it follows ${parameter+alt_value} pattern: If parameter set, use alt_value, else use null string. See more on it here.

So now, we can put all pieces together: when shell executes line 2, perl program (i.e. perl5) will be invoked and execute the script itself (expand from $0) and supply the argument list, which may be required by the script.

Example

Let me give an example.

Suppose we have a perl script named foo:

#!/usr/bin/perl
eval 'exec /wsdb/oemtools/linux/bin/perl5.16.2 -S $0 ${1+"$@"}'
    if 0;

use Config;
my $perl = $Config{perlpath};

print $perl."\n";

Besides the daunting code chunk, the rest will print out the absolute path of perl interpreter that our script get executed by. Now, let’s try out different way of executing our perl script foo:

$ perl foo
/usr/bin/perl
$ ./foo
/usr/bin/perl
$ sh foo
/wsdb/oemtools/linuxbin/perl5.16.2

The first two cases, we run the perl script in a standard way, since my system (SUSE Linux 11) supports shebang, the script gets executed by the perl interpreter specified in the shebang line. However, if we try to mimic the system that doesn’t support shebang by executing our script using shell (i.e., sh), the script is also get interpreted as perl script but with the perl interpreter from eval part. Notice sh usage here, sometimes the user of the script may assume the script is written by shell, and they will try to execute the script by sh. Then, in this case, our daunting code chunk provides a defensive mechanism that allows the perl script to be executed correctly even when it is run by shell.

With this explanation, I don’t think code chunk in find2perl will be daunting to you now:

#! /usr/bin/perl -w
    eval 'exec /usr/bin/perl -S $0 ${1+"$@"}'
        if 0; #$running_under_some_shell

 Modern days

Nowadays, people rarely use that daunting code chunk solely because some systems don’t support shebang. Increasingly, that daunting code chunk usually appears when people want to use a specified version of perl (not system default one like /usr/bin/perl) and at the same time, maintain some portability to the system that doesn’t support shebang. However, if we solely consider to use a specified version of perl instead of default one, then there is more than one way to do so. My list may not complete. Please feel free to comment below if I miss some usage on invoking customized perl.

#!/usr/bin/perl
eval 'exec /wsdb/oemtools/linux/bin/perl5.16.2 -S $0 ${1+"$@"}'
    if 0;

The first way is again our daunting code chunk. If we run our script by sh, then our customized perl (i.e., /wsdb/oemtools/linux/bin/perl5.16.2) is executed. -S as perl command option used here is to make perl use PATH environment variable to search for the script because on some system $0 doesn’t always contain the full pathname to the script. You can read more about -S option in perlrun doc and in fact, the daunting code chunk also got explained there.

The second way is to put the following code at the first line of perl script:

#!/wsdb/oemtools/linux/bin/perl5.16.2

This way you directly hardcode the customized perl interpreter in your script. This may sacrifice portability of the script.

Another way to use customized perl interpreter is put this code chunk, again, at the first line of the script:

#!/usr/bin/env perl

This will tell the system (that understands the shebang) to find the first “perl” executable in the list of $PATH. If you want to run your customized perl interpreter this way, you want to put the path to your customized perl interpreter at the beginning of $PATH environment varaible so that you ensure if first “perl” executable found by the system from $PATH is indeed the perl interpreter you want to use.

The last way to run your customized perl interpreter is somewhat similar to our daunting code chunk but with significant difference:

#!/bin/sh
#! -*-perl-*-
eval 'exec /wsdb/oemtools/linux/bin/perl5.16.2 -x -wS $0 ${1+"$@"}'
    if 0;

Let’s run it first to see what we can get. Like previous example section, we put the above code chunk inside a script called bar:

#!/usr/bin/sh
#!-*-perl-*-
eval 'exec /wsdb/oemtools/linux/bin/perl5.16.2 -x -wS $0 ${1+"$@"}'
    if 0;

use Config;
my $perl = $Config{perlpath};

print $perl."\n";
$ bar
/wsdb/oemtools/linuxbin/perl5.16.2
$ perl bar
/wsdb/oemtools/linuxbin/perl5.16.2
$ ./bar
/wsdb/oemtools/linuxbin/perl5.16.2
$ sh bar
/wsdb/oemtools/linuxbin/perl5.16.2

No matter how we execute our script, we always use our customized perl interpreter, even when system perl is explicitly specified (i.e., perl bar). The significant difference than our original daunting code chunk is the use of -x option. The -x does the following:

tells Perl that the program is embedded in a larger chunk of unrelated text, such as in a mail message. Leading garbage will be discarded until the first line that starts with #! and contains the string “perl”. Any meaningful switches on that line will be applied.

Let me walk through what exactly happen in our case. We will use the following information taken from perldoc during the walkthrough as well:

 If the #! line does not contain the word “perl” nor the word “indir”, the program named after the #! is executed instead of the Perl interpreter. This is slightly bizarre, but it helps people on machines that don’t do #! , because they can tell a program that their SHELL is /usr/bin/perl, and Perl will then dispatch the program to the correct interpreter for them.

We launch our bar script as ./bar:

1. Shell executes our script ./bar
2. The system actually executes /bin/sh ./bar because of our shebang specification.
3. sh executes /wsdb/oemtools/linux/bin/perl5.16.2 -x -wS bar
4. /wsdb/oemtools/linux/bin/perl5.16.2 skips:

#!/usr/bin/sh
#!-*-perl-*-
eval 'exec /wsdb/oemtools/linux/bin/perl5.16.2 -x -wS $0 ${1+"$@"}'
    if 0;

and executes:

use Config;
my $perl = $Config{perlpath};

print $perl."\n";

Let’s break down this step into further detail:

4.1 /wsdb/oemtools/linux/bin/perl5.16.2 executes /usr/bin/sh ./bar because it sees a shebang that doesn’t contain the word “perl”
4.2 eval part get executed (i.e., sh executes
/wsdb/oemtools/linux/bin/perl5.16.2 -x -wS bar
)
4.3 Since -x is specified, the first line #!/usr/bin/sh is ignored because it is a shebang but doesn’t contain the string “perl”. Line 2-3 is ignored because if 0. So, the execution starts with use Config; and move forward.

Let’s try launch our bar script using perl bar to see why system perl is not used in this case:

1. Shell executes the script perl bar
2. perl (i.e. /usr/bin/perl) executes /bin/sh bar because it sees a shebang that doesn’t contain the word “perl”
3. eval part get executed (i.e., sh executes
/wsdb/oemtools/linux/bin/perl5.16.2 -x -wS bar
)
4. So our script bar is executed by /wsdb/oemtools/linux/bin/perl5.16.2 instead of /usr/bin/perl

Let’s Practice

Based upon what we learn, you should not have much trouble understanding why

#!/bin/sh
eval 'exec /wsdb/oemtools/linux/bin/perl5.16.2 -wS $0 ${1+"$@"}'
    if 0;

will lead to

/bin/sh: -S: invalid option

error. The key lies in we are not using 1) shebang + string word “perl” and 2) -x option. If you have hard time finding out why, here is the answer.

Thanks for the reading!

 Reference

Advertisements

Book Review: There Is Nothing Wrong with You: Going Beyond Self-Hate

51gdrnmq2ol-_sx321_bo1204203200_

这本书其实我在4,5月份的时候就已经读完了,在从5月份到7月份这期间,甚至我还拿这本书做过一次Toastmaster club speech。如果你有兴趣,这是我当时做演讲用到的slides。之所以拖到这个时间才去写,原因主要有二:

  • 之前两个月我忙着去考GRE
  • 像这种功能书,如果刚读完就去写的话,难免会有概率误人子弟。为啥呢?如果我刚放下书就跑到这里胡扯,而不是亲身按照书上的方法去实践一下的话,这不是误人子弟还能是啥?!(那你为啥还去做演讲呢?这看到本好书难免会激动要去和别人安利一番嘛)
  • 现在时间刚刚好。自己实践了两个月,可以稍微有资格去评价一下书上的观点;脑子里记得关于这本书的感想也行将淡去,是时候付诸文字记录一下了。

缘起

做程序员的多多少少会有一些完美主义。作为土生土长的中国人,这点可能会更为强烈一些。在只许成功不许失败的家庭教育下,在社会上编程大神日益成为网红的时代潮流下,我,一个还算有点上进心的中国程序员,难免会有一些焦虑,自责,乃至自我厌恶——you suck because you’re bad.

终于在某一天的晚上,出于一个记不清的理由(估计是又遭到老妈针对我某件事做法上的批评),我自我厌恶情绪达到了顶点。同时,另外一个从来没出现过的声音出人意料地也在我脑袋里响了起来——老子今晚就要以暴制暴,好好处理一下内心中另外一个我。所以,我想到了之前在reddit上看到的一个post: [Text] Self- loathing is poison. It leads to escapism, procrastination and and depression. Stop focusing on what holds you back. 并想从中获取一些对抗自我厌恶的手段。于是,我就发现了这本书。

详细谈谈这本书

其实你如果不抱着一颗打破沙锅问到底的心态去对self-hate进行全方位剖析的话,这本书给出的针对self-hate的解决之道非常简单,那就是meditation,也就是中文的冥想。如果你能定期(每天一次或者一周一次)去坐下来,慢慢得去聆听自己的呼吸,很多东西就会不然自明。换句话说,你完全可以不用去读这本书,只要去冥想,去接近自己的内心,慢慢的你就会和自己平常所感受到的self-hate达成一种平衡。你就会慢慢的去接受self-hate,接受他的存在,进而做到与之共存,最后将他作为你自己的一部分慢慢吸收进来。正如书中讲到的:

You could just sit down and face a wall and eventually you would understand all of this (what is self-hate, how self-hate is formed …). It’s all available without having any intellectual understanding of it.

但是,如果一本书这样去写,肯定不会有人去读它。因为我们现在处于一个快节奏的社会,人们更希望把时间分配到能立竿见影的事物上面去,如果说一本书告诉人去做冥想就能摆脱self-hate,但并没有告诉人们为什么这个方法会有效的话,我相信很少有人会抱着一种怀疑的态度将书中的方法坚持到底的。所以,书的作者从self-hate的形成原因入手,慢慢地引出mediation,使读者在读完整本书后会有一种完整体系的感觉,从而有信心和动力去贯彻书中所提到的方法。

再细节的地方我不想一一去谈。我就说说这本书给我留下印象比较深的地方。

字体,排版,与插图

下面这张图是我随机从书中挑出来的一页。jfszhxd说实话,我上一次看到这种字体,这种排版,还有这种插图的是在我小学和幼儿园的时候。所以,这本书给我一种亲切,童真的感觉。一种我只要翻翻这本书就会减少很多烦恼的感觉。这也使得我在读这本书的两个月来,每天上班坐上地铁,就会迫不及待的去打开这本书去读。读的时候,书中的字体并没有给人一种严肃说教的感觉。相反的是,你会感觉是一个非常熟识的朋友在一个非常放松的环境下,给你提出的一些真诚的建议。从这个角度上讲,这本书可以非常快得抓住读者的心。

 

 

看问题角度的改变

这本书给我了一种不同的角度去看待self-hate。而看似复杂困难的问题,往往因为角度的转变,解决起来也会变得迎刃而解。比如说self-hate的产生。我过去认为self-hate是一种追求完美的体现,但是我从来没有去深究,self-hate的产生是否是由于追求完美。但是这本书告诉我:self-hate的产生是因为童年并没有被满足的需求。

举个例子来说,假如泽远小的时候特别想要台电脑,但是当我这个请求向父母提出的时候,我得到的回复是”NO”。这个时候,在我的潜意识里就会得到一种判断,那就是: 我的请求之所以没有被父母采纳,是因为我本身不够好,如果我做的更好的话,父母就会给我买电脑。在那个时候,我并没有意识到父母有可能存在问题的可能性(比如他们不给我买电脑并不是因为我做的不够好)。在我的眼里,他们就是权威,而且我的生存也强烈依靠他们。因此,他们不给我买电脑肯定是因为我自身的问题。因此我决定通过不断指责我自己的方式来“追求完美”,从而希望我的需求最后能被父母满足。这种追求完美的心态就渐渐促成了self-hate。

当书中将这件事讲明白的时候,我就突然意识到我平时的一些默认的假设并不是“神圣而不可侵犯”的。比如说,我是否得到父母的认可其实和我本身的完美程度往往并没有直接联系。换句话说,我脑海里越有希望得到他们的认可的这种期望却又得到满足的时候,我下意识就会说我确实不够好,如果我再自己变得完美一些的话,父母就会认可我。可是,我并没有意识到也许父母他们本身并没有存在通过语言等方式去表扬你认可你这种意识。这和你本身是否完美并无关系。即使你做得再好,如果他们没有意识去通过直接的方式认可你的话,你还是依然会陷入自己不够好的死循环中。

明白了这一点,我也就渐渐学会去和父母处关系了。过去,我往往想要指出他们的不足,指出他们应该去多多表扬我,认可我。可是当他们还是保持原有的默不作声的话,我就会感到焦躁。可是,上面这个例子告诉我,我不一定要通过父母去满足我这种需求。比如说我将“你是最好的!”这句话录下来,每天放给自己听。听这句话的本身其实就是一种满足自我需求。而当自我需求被满足后,上面提到的死循环才有可能会被打破。

再引申一步讲,遇到任何困难,比如说表白被拒,找工作不顺,这些往往并不是因为你自己有问题,而是因为这些事本身就是有难度的。换句话说,就是Doing bad at something is not because you’re bad. It is because the thing you are trying to do is hard in itself. 因此,换个角度,就有可能以一个更加积极的态度去看待问题。而不是始终被self-hate所控制,暗无天日,陷入一种消极的状态中。

Too obvious to ignore

我常常会发现有些东西明明摆在那里,但是往往因为它太简单,太不需要去刻意去观察,我就会去忽略它。而被忽略的这个事物却往往会成为解开心结的关键。

书中举了一个非常生动的例子。假如你突然有一天想开始跑步,这时候突然有一个声音蹦出来叫道:“你穿成这样就跑步?你居然敢声称你要跑步。。。” 同时,又有另一个声音向你说道:“你已经做的很好了。就这样跑下去。你跑得非常好,继续加油!”。你仔细想想哪种声音是真正想让你跑步的?而哪种声音是装作给你提建设性意见 让你跑步,但其实是在阻挠你的?

没错,self-hate往往就是以一种建设性意见的姿态出现在你的内心,假装是为了你好,实际上却是在万般阻挠你前进。再试想一下,如果这两种声音不是出自你的心声,而是出自你的两个朋友的话,估计你已经早已离开那个不停向你提出“建设性意见”的朋友远远的了。可是,为什么我们不能对self-hate采取同样的办法呢?仅仅因为self-hate是存活在我们的内心,并没有具体的形态,我们就可以对他听之任之了吗?! 不!

讲到这里,你应该多多少少感受到这本书独特的切入点和看事情的角度了吧。

再多说一句。对于你总给你提“建设性意见”的父母,就因为他们是你的父母,你就应该对他们的意见不假思索的全盘接受了吗?不,因为他们是被他们提“建设性意见”的父母抚养长大的。所以,他们并不知道除了提“建设性意见”,还有什么样的方式来帮助你。现在仔细想想,他们所谓的“建设性意见”有多少是帮助你在人生道路上奔跑,又有多少反倒是泄了你的气,让你每走一小步都觉得无比艰难呢?同理,我曾经幻想去改变我的父母这一点,但是,正如书中所言,如果他们能改,他们早就已经改掉了。只有当我意识到这一点,我反倒能放下我的抵触情绪,以一种包容的心态去看待他们的“建设性意见”。因为我知道,他们是可怜的,他们是被他们的父母以“建设性意见”这一self-hate的表现形式所抚养长大的。

这本书像这样的例子还有很多。我就不一一列举了。最后,我想说,如果你也面对着像我一样被self-hate搞得痛不欲生的话,看看这本书,哪怕你不坚持做meditation,这本书也会改变你看待事物的方式,从而让你获得心灵上的解脱。