
Sunday, August 30, 2009
Tuesday, August 18, 2009
Amazon gripe
OK, let me start by saying that I'm a total Amazon fanboy, or to be more specific I'm a total AWS fanboy. They've got some great services out there (queue manager, SDB, S3, EC2) with excellent documentation, and they also have a _lot_ of functionality in their APIs for their store, along with pretty good documentation - my only complaint about the documentation is that sometimes it's pretty tough to find what you need.
Amongst many, many other APIs, AWS has an API to get info for a particular ASIN. Every item in Amazon has an ASIN, which is a unique identifier for that specific product. You can actually just enter it in the search bar if you want to find something quick and you happen to know its ASIN (like, for example, if you want to see the info for "Wang Dang Doodle" by Livin' Blues, just pop B000008652 into the Amazon search field).
Combine this with Musicbrainz's XML Webservice (which often has the ASIN for an album as part of its metadata) and a lot of elbow grease, and you theoretically have a relatively quick way to look up all those one-hit tracks that you ripped or downloaded 10 years ago, before people were concerned about tagging or filename conventions or any of those nuisances.
There are other uses, too; unless the album is quite rare (or the song was only released as a single, as often happens with older tracks), Amazon will probably have a decent-quality picture of the album art. I use iTunes for everything, and one irritating thing about iTunes for me is that when you use iTunes to rip a CD, then get the album art, the album art is stored in iTunes and not in the files themselves. So, if you migrate those albums somewhere else, or (as I have done once or twice), screw up something to the point where you have to delete your whole iTunes DB and rebuild it, the album art is gone. Unacceptable! I want my album art embedded in my files dammit!
So, anyway, I've written a lot of scripts to search and tag songs / albums based on what's available in Musicbrainz and / or Amazon. I've gotten past all the easy ones, and now I'm onto the really weird, hard stuff like Hoots Mon and am having to manually search Amazon / Wikipedia / whatever for just about everything.
Anyway, I digress (seriously!). The point is that I'm a huge fan of AWS, but I'll get to the gripe. AWS recently added a requirement that all queries to AWS be digitally signed with your AWS account secret. I received warning emails for about 2 months before they made the official switch, so I can't say I wasn't warned, although when I skimmed the emails I thought it was only for advertising calls, which makes sense (to prevent abuse).
Now, the process of signing a REST query isn't very difficult, but it's a really big pain in the ass; you have to deconstruct the query, add a timestamp, sanitize it, re-order it, build a hash from it and your secret, base64 the hash, add the hash to the arguments, and re-order it again. Not rocket science, but it still took me a good 2 hours to write subroutines to do it. Which makes me wonder - why on earth doesn't Amazon distribute modules for the main languages (Perl, PHP, Ruby, Java, maybe Python, and whatever the IIS folks are using nowadays), or at least C source or Java byte code to do this? Pretty much every developer who works against Amazon APIs has had to do the exact same thing that I have, and if Amazon changes the signing process, then all of us are going to need to re-write our code. That's ridiculous!
Anyway, I'm writing my own damn Perl module (already have the subroutines, just need to clean it up and make it OO), and if I end up having sufficient free time I'll submit it to CPAN. Even though my code will be crap, maybe someone will use it as a jumping point to make it better. But, I ask again, why the hell hasn't Amazon done this?
Amongst many, many other APIs, AWS has an API to get info for a particular ASIN. Every item in Amazon has an ASIN, which is a unique identifier for that specific product. You can actually just enter it in the search bar if you want to find something quick and you happen to know its ASIN (like, for example, if you want to see the info for "Wang Dang Doodle" by Livin' Blues, just pop B000008652 into the Amazon search field).
Combine this with Musicbrainz's XML Webservice (which often has the ASIN for an album as part of its metadata) and a lot of elbow grease, and you theoretically have a relatively quick way to look up all those one-hit tracks that you ripped or downloaded 10 years ago, before people were concerned about tagging or filename conventions or any of those nuisances.
There are other uses, too; unless the album is quite rare (or the song was only released as a single, as often happens with older tracks), Amazon will probably have a decent-quality picture of the album art. I use iTunes for everything, and one irritating thing about iTunes for me is that when you use iTunes to rip a CD, then get the album art, the album art is stored in iTunes and not in the files themselves. So, if you migrate those albums somewhere else, or (as I have done once or twice), screw up something to the point where you have to delete your whole iTunes DB and rebuild it, the album art is gone. Unacceptable! I want my album art embedded in my files dammit!
So, anyway, I've written a lot of scripts to search and tag songs / albums based on what's available in Musicbrainz and / or Amazon. I've gotten past all the easy ones, and now I'm onto the really weird, hard stuff like Hoots Mon and am having to manually search Amazon / Wikipedia / whatever for just about everything.
Anyway, I digress (seriously!). The point is that I'm a huge fan of AWS, but I'll get to the gripe. AWS recently added a requirement that all queries to AWS be digitally signed with your AWS account secret. I received warning emails for about 2 months before they made the official switch, so I can't say I wasn't warned, although when I skimmed the emails I thought it was only for advertising calls, which makes sense (to prevent abuse).
Now, the process of signing a REST query isn't very difficult, but it's a really big pain in the ass; you have to deconstruct the query, add a timestamp, sanitize it, re-order it, build a hash from it and your secret, base64 the hash, add the hash to the arguments, and re-order it again. Not rocket science, but it still took me a good 2 hours to write subroutines to do it. Which makes me wonder - why on earth doesn't Amazon distribute modules for the main languages (Perl, PHP, Ruby, Java, maybe Python, and whatever the IIS folks are using nowadays), or at least C source or Java byte code to do this? Pretty much every developer who works against Amazon APIs has had to do the exact same thing that I have, and if Amazon changes the signing process, then all of us are going to need to re-write our code. That's ridiculous!
Anyway, I'm writing my own damn Perl module (already have the subroutines, just need to clean it up and make it OO), and if I end up having sufficient free time I'll submit it to CPAN. Even though my code will be crap, maybe someone will use it as a jumping point to make it better. But, I ask again, why the hell hasn't Amazon done this?
Friday, August 14, 2009
Combo locks
I had a few old Master combination locks lying around and decided to retrieve the combinations as described here. Being a geek, I wrote a script to help me keep track. Using this I was able to get all combos at less than 10 minutes per lock. Of course, it took me a good 20-30 minutes to write the script, but that was more fun than spinning a combination lock.
Also, this is far from an advanced shell script, but the nice thing about it is that it's quick, dirty, and easy. You can actually do a lot with bash very quickly once you get to know it, and one nice thing about bash vs. Perl is that it's really easy to get a gut feeling when bash isn't going to cut it and Perl is the way to go, for example when you need to parse a huge amount of data or do some really complicated regex work.
Anyway, if this helps you, then great, if not then at least it's documented somewhere...
[Edit: sorry if feed readers got this multiple times, I had to fix a couple of typos...]
##########START#############
#!/bin/bash
# Bash script to print out all likely combos for a master combo lock. Rough and ready but it works
# Written by Nate Aiman-Smith (http://staunchtech.blogspot.com). To be run as-is by a non-priviliged user.
## Variables
# One argument - the last number.
# See http://www.fusor.us/lockpick.html for a simple explanation on how to get it
thirdnum=$1
# File to keep track of tried combos. Replace with your preferred place if required
trackfile="/tmp/triednums_$thirdnum"
# Difference between first and second numbers that we'll print out - more than 24 is, in my very limited experience, unlikely.
# With more practice could probably get a better number.
fastdistance=24
## Functions
verify_nums() {
num=$mod
while [ $num -lt 40 ]
do
if [ $num=$thirdnum ]
then return 0
fi
num=`expr $num '+' 4`
done
return 1
}
print_valid_firstnums() {
num=$mod
while [ $num -lt 40 ]
do
echo -n "$num "
num=`expr $num '+' 4`
done
echo
}
print_combos() {
mod=`expr $firstnum '%' 4`
mod=`expr $mod '+' 2`
mod=`expr $mod '%' 4`
num=$mod
while [ $num -lt 40 ]
do
# Swap the two lines if you want to get the _unlikely_ combos...
if check_distance
#if ! check_distance
then echo "$firstnum $num $thirdnum"
fi
num=`expr $num '+' 4`
done
}
# To facilitate, let's try to stick to second numbers that are
# less than $fastdistance away from the first
check_distance() {
secondnum=$num
# Equal numbers always fail
if [ $secondnum -eq $firstnum ]
then return 1
elif [ $secondnum -lt $firstnum ]
then
secondnum=`expr $secondnum '+' 40`
fi
diff=`expr $secondnum '-' $firstnum`
if [ $diff -gt $fastdistance ]
then
return 1
fi
return 0
}
##Main
if [ ! $thirdnum -lt 40 ]
then
echo "invalid number"
exit 1
fi
if [ -f $trackfile ]
then
echo "Found tracking file $trackfile - hit ctrl-C, delete the file, and start over if you want to start fresh"
fi
mod=`expr $thirdnum '%' 4`
for valid in `print_valid_firstnums`
do
if ! grep "^$valid" $trackfile &> /dev/null
then
echo "Combinations for $valid:"
firstnum=$valid
print_combos
echo "Press return when done, hit ctrl-C if you bail"
read blankline
echo $valid >> $trackfile
fi
done
############END################
Also, this is far from an advanced shell script, but the nice thing about it is that it's quick, dirty, and easy. You can actually do a lot with bash very quickly once you get to know it, and one nice thing about bash vs. Perl is that it's really easy to get a gut feeling when bash isn't going to cut it and Perl is the way to go, for example when you need to parse a huge amount of data or do some really complicated regex work.
Anyway, if this helps you, then great, if not then at least it's documented somewhere...
[Edit: sorry if feed readers got this multiple times, I had to fix a couple of typos...]
##########START#############
#!/bin/bash
# Bash script to print out all likely combos for a master combo lock. Rough and ready but it works
# Written by Nate Aiman-Smith (http://staunchtech.blogspot.com). To be run as-is by a non-priviliged user.
## Variables
# One argument - the last number.
# See http://www.fusor.us/lockpick.html for a simple explanation on how to get it
thirdnum=$1
# File to keep track of tried combos. Replace with your preferred place if required
trackfile="/tmp/triednums_$thirdnum"
# Difference between first and second numbers that we'll print out - more than 24 is, in my very limited experience, unlikely.
# With more practice could probably get a better number.
fastdistance=24
## Functions
verify_nums() {
num=$mod
while [ $num -lt 40 ]
do
if [ $num=$thirdnum ]
then return 0
fi
num=`expr $num '+' 4`
done
return 1
}
print_valid_firstnums() {
num=$mod
while [ $num -lt 40 ]
do
echo -n "$num "
num=`expr $num '+' 4`
done
echo
}
print_combos() {
mod=`expr $firstnum '%' 4`
mod=`expr $mod '+' 2`
mod=`expr $mod '%' 4`
num=$mod
while [ $num -lt 40 ]
do
# Swap the two lines if you want to get the _unlikely_ combos...
if check_distance
#if ! check_distance
then echo "$firstnum $num $thirdnum"
fi
num=`expr $num '+' 4`
done
}
# To facilitate, let's try to stick to second numbers that are
# less than $fastdistance away from the first
check_distance() {
secondnum=$num
# Equal numbers always fail
if [ $secondnum -eq $firstnum ]
then return 1
elif [ $secondnum -lt $firstnum ]
then
secondnum=`expr $secondnum '+' 40`
fi
diff=`expr $secondnum '-' $firstnum`
if [ $diff -gt $fastdistance ]
then
return 1
fi
return 0
}
##Main
if [ ! $thirdnum -lt 40 ]
then
echo "invalid number"
exit 1
fi
if [ -f $trackfile ]
then
echo "Found tracking file $trackfile - hit ctrl-C, delete the file, and start over if you want to start fresh"
fi
mod=`expr $thirdnum '%' 4`
for valid in `print_valid_firstnums`
do
if ! grep "^$valid" $trackfile &> /dev/null
then
echo "Combinations for $valid:"
firstnum=$valid
print_combos
echo "Press return when done, hit ctrl-C if you bail"
read blankline
echo $valid >> $trackfile
fi
done
############END################
Subscribe to:
Posts (Atom)