Awk and sed

For those of you non GNU-junkies, awk and sed are an essential part of any admin's toolbox.  They are interpreted languages (meaning there's no compiling involved and your programs just run on the fly) and they've been around for a long long time.  It's pretty rare that I write a script of more than 20 lines and don't use at least one of them.

Awk and sed are pretty feature-heavy languages, and in fact there's an entire O'Reilly book just on those two alone.  However, I really only use them for one thing each; I use awk to extract a particular field from a string, and I use sed to do quick and dirty substitutions.   So, here's the Nate Aiman-Smith quick and dirty howto:

For awk, 99% of the time I use this line:

[command printing out stuff] | awk -F'[field separator]' '{print $[field number]}'

That's it!  So, for example, if I had $numbers (or %numbers% for you windows guys) set to  "one two three four" and I wanted to get the third word, I would just do:

echo %numbers% | awk '{print $3}'

which would print out "three".  Note that the field separator defaults to whitespace.  If my string was "one::two::three::four", I could do:

echo %numbers% | awk -F'::' '{print $3}'

For the same result.

For sed, the construction is:

sed 's/[search]/[replace]/' [file]

or 

[command printing out stuff] | sed 's/[search]/[replace]/'

That does the substitution once (the first time the search string is found).  You can add '/g' at the end to make it global.  So, for example, to change all instances of "Nathan" to "Naton" in my file "recommendation.txt", I could do:

cat (or "type" for you windows folks) recommendation.txt | sed 's/Nathan/Naton/g' > good_recommendation.txt

Which would create a new file, good_recommendation.txt with the substitutions.

Now of course, Perl can do all these things almost as easily, and it can do it a _lot_ faster, but when you're writing the quick and dirty script or one-liner, it's a lot easier just to use one of these.   If you're a Unix administrator, you need to know at least these basics of sed and awk, because you're going to see them a lot.  It saddens me how many Linux admins don't seem to know these basic tools, but on the other hand it makes me look like some kind of god when I'm able to do a search-and-replace on 1000 files in about 2 minutes with a one-liner.

There are actually precompiled windows binaries available for them if you want to start working on your own script-fu.  Or, even better, use Cygwin or a Linux distro (or open a terminal in Mac OS X) to really dig into it.

the 4th amendment

So, Clau and I were in Boston last weekend, we wanted to go to a particular Italian restaurant that's apparently the most popular Italian restaurant in Boston, on the most popular night for Italian food of the year. We were stuck in a majorly long line and got to conversing with the folks near us (somewhere in their early 50s by the look of them) - turned out they were major Republican types from Naples, FL. I started talking to the husband while Clau got into a conversation with the wife. Of course, us living in NYC, 9/11 comes up (I didn't bring it up!) and the guy starts talking about how quickly we've forgotten about it and started acting like the whole thing never happened, so I had two points for him:
1) Maybe you've forgotten, but in NYC the shadow of that day hangs over the city non-stop like a cloud
2) If by "forgetting" you mean picking ourselves up and getting on with our lives, then yeah, that's pretty much what NYC has done, and maybe the rest of the country should too. The whole point of terrorism isn't to kill a few thousand people; it's to put fear and chaos into the entire nation, and that's exactly what's happened. If you really want to defeat terrorism, refuse to be terrorized.
As an example of how bad it's gotten in NYC, I pointed out that we have random subway searches where I don't think the NYPD has any expectation of actually catching any terrorists (see my earlier post on this for an anecdote), but rather it's for security theater and to catch some other offenders. The way I see it, they're searching people who aren't really aware of their civil rights, and if they find, for example, a gram of cocaine in a woman's purse, they're not going to say "well, this is obviously not a weapon - here's your coke back, have a nice day".
To which the fellow from Florida responded "Well, what's wrong with that? A person shouldn't be walking around with cocaine in the first place - if the cops catch some druggies instead of terrorists, isn't that still a good thing?"
To which I responded: it's the principle of the thing - we have a fundamental right against that exact type of search, and people are giving it up.
To which he again responded: well, what's the point of that? If you have nothing to hide, what's the problem?
And I didn't really have an answer - I mean, I started to answer, but I ended up sounding more and more like a privacy nut. I ended up saying something like "well, it seems that there are two extremes: anarchy and totalitarianism. Most people's ideal falls somewhere between those two extremes, and I guess it's a matter of where you think the line should be drawn. For me, using terrorism as a reason to search people's bags on the subway is too far left of that line. If you don't agree, then I guess that's your prerogative, but you should decide where you would draw that line." Of course, I was flustered so I didn't put it quite that eloquently, but I think I got my point across.
Yet now I find myself wondering: why should reasonable people be opposed to these searches? I mean, I've been objecting on the pure basis that it seems a violation of a fundamental American right, but it seems that the whole point of the amendment was a response to overly generalized warrants, rather than police or government randomly stopping folks in the street and making them empty their pockets. Of course, it seems that the whole system of needing a warrant in the first place was a fundamental part of English law at the time, and the 4th amendment was something of a refinement of warrant requirements and applicability.
For the record, I'm still against NYPD subway search stations, but now I'm having a tougher time rationally explaining it to myself.  I guess it comes down to the ability of a government to oppress its citizens, with the underlying idea being that the government will use and abuse any power given to it, rather than using that power to serve the people.

This has been rolling around in the back of my head for a little while - I just wanted to get it down on paper.  I'll be sure to have a long talk with my dad about it the next time I see him.

Using a UPS as a transformer

So, as part of the Munich prep, I've been wondering how I can deal with my various American devices. We've been encouraged to leave behind our glorious TV (which I'm not sure I agree with, but then again our current unit is getting a little long in the tooth), but of course I will still have our wifi router, my Wii, and chargers for various devices.
Everyone seems to say that you need a step-down transformer in order to deal with the 220v power in Germany, but I've been wondering why I can't just use a UPS that can take 220v in and put 110v out. Theoretically, a UPS has two inverters - one leading into the battery and another leading out, so in theory (from my extremely limited knowledge of electronics) I should be able to use this in place of a transformer, with the side benefit of getting power redundancy and conditioning. So, I hit Google just to confirm, and I haven't been able to find one single account of anybody doing this. So now I have to face the choice of spending (and possible throwing away) $150-$200 on a decent dual-voltage UPS, or just dealing with it over there (which will undoubtedly cost more). But what really weirds me out is my inability to find a single account or testimonial of this on Google.