MagiCal’s Latest Version

Recorded on April 22nd, 2014

One of the things I have been doing for the App Data Info project is trying to figure out how to find the latest version of (non-Mac App Store) Mac apps.

If the app uses Sparkle, it’s pretty easy because the update feed is XML and easy to parse. It goes something like this:

curl -sfL https://flexibits.com/fantastical/appcast.php |\
tr ' ' '\012' |\
awk -F'"' '/sparkle:shortVersionString/{print $2}' |\
head -1

Yes, I know I could get away with not using tr but I prefer this way.

Ideally you can find the Sparkle feed just by looking in the “Info.plist” of the app. Here’s a quick way to get most1 of them on your Mac:

egrep  '<string>http(s)?://' /Applications/*.app/Contents/Info.plist |\
tr -d '[:blank:]' |\
sort -u

Some developers have Sparkle feeds (or similar) but they make it very difficult to find them. No, I don’t know why either. Fortunately I have been able to find most of them through various amounts of cleverness and the occasional use of Charles Web Debugging Proxy. Sometimes you have to use a special User Agent when accessing the feed in order to get the right information for a specific app. Also understandable, especially if you have several apps and want to use one feed.

So far the most challenging one has also been the most futile, because it is for MagiCal, an app which I don’t expect to ever be updated.

curl --fail --silent --header 'Content-Type: text/xml' \
--data '<?xml version="1.0"?><methodCall><methodName>application.getLatestVersion</methodName><params><param><value>MagiCal</value></param></params></methodCall>' \
'http://www.charcoaldesign.co.uk/rpc'

That will return an XML formatted “change log” for the most recent version, which includes the version number:

<?xml version="1.0" encoding="UTF-8"?>
<methodResponse>
  <value>
    <struct>
     <member>
       <name>version</name>
       <value>1.1r4</value>
     </member>
     <member>
       <name>notes</name>
       <value>
        <array>
          <data>
            <value>[fix] Icon no longer appears in Dock (introduced in 1.1r3)</value>
            <value>[fix] Clicks are now detected anywhere on the menu item area, not just the text or icon.</value>
            <value>[fix] Fixed custom colour option.</value>
          </data>
        </array>
       </value>
     </member>
    </struct>
  </value>
</methodResponse>

Now that we know how to get the information from the server, we need to pare it down to just what we want. There are different ways to do this, but I tend to use either one of two:

  1. remove all of the line feeds (tr -d '\012') so that you get one long string of text, and then use sed to remove the parts you don’t want

  2. Use fgrep to pull out just the line(s) that you want.

In the above example, what we want is the <value>1.1r4</value> line, so how do we get it? Well, we can’t just look for <value> because there are lots of lines with that. What we really want is the line after <name>version</name> which we can get easily using fgrep -A1 '<name>version</name>' which tells fgrep to look for <name>version</name> and show one line after it —- that’s the purpose of the -A1. If you wanted one line before it you’d use -B1. If we do that, we’ll get this:

<name>version</name>
<value>1.1r4</value>

Now we can do something pretty slick using awk which is a really powerful program that I only know a very little about, but I know enough to make it useful to me in this case. We want what is between the <value> and </value> so we can tell awk to split the lines whenever it finds either a < or > by using

awk -F'>|<' 

The -F tells awk that we want to use whatever comes after it to mark fields.

The '>|<' tells awk to use “either < or >

The | (aka “pipe”) means “or”.

we need to tell awk to look for the word value and give us the third field in the line that includes the word value:

'/value/{print $3}'

Put that all together and you get:

awk -F'>|<' '/value/{print $3}'

$3 means “third field” in this case. The other handy one to know is $NF which means “the last field” regardless of how many there are.

TextExpander Tip: I use variations on that awk command all the time, and I constantly mistype awk as awf so I made a TextExpander shortcut called awf:

awk -F'%|' '//{print $x}'

(I put that shortcut in a restricted group so it only expands in iTerm, BBEdit, and CodeRunner, so it won’t accidentally expand when I’m trying to type the word “awful” or similar.)

Ready for the whole thing?

Here’s the entire command we’ve been building, all put together:

curl --fail --silent --header 'Content-Type: text/xml' \
--data '<?xml version="1.0"?><methodCall><methodName>application.getLatestVersion</methodName><params><param><value>MagiCal</value></param></params></methodCall>' \
'http://www.charcoaldesign.co.uk/rpc'|\
fgrep -A1 '<name>version</name>' |\
awk -F'>|<' '/value/{print $3}'

“Run this curl command, narrow it down with fgrep, and then filter it with awk.”

Someone who knows awk better than I do could probably do this without using fgrep but for my purposes, this works fine, it’s easy for me to read and understand, and the difference between one fgrep call and none is practically inconsequential.


  1. plist files can either be XML or binary. If you want to run egrep on one a binary plist, you’ll need to run it through plutil first: plutil -convert xml1 -o - /path/to/Info.plist which will leave the original file alone but send the XML version to stdout

Removing DRM from iTunes Video

Recorded on April 20th, 2014

The great irony of being a paying customer of the TV and movie industry is that you end up with a worse product. Put in a DVD or Blu-Ray, and what happens? Ads and previews come up, which are either “unskippable” or difficult to skip. Then the FBI/Interpol warning. You know who doesn’t have to put up with that crap? People who don’t pay for movies or TV shows, but who download them from the Internet for free.

That’s a pretty lousy way to treat your customers, but the switch from DVD to Blu-Ray has shown that “Hollywood” wants more control, and they will use that control to put more crap between you and your ability to watch content that you’ve paid for.

One of the few decent alternatives is to buy movies from iTunes. They don’t come with the unskippable menus and other crap (I assume that Hollywood wanted them and Apple refused, but I don’t know that for a fact). However, there are still some frustrating restrictions.

Want to take a screenshot of a scene in a movie? Sorry, you can’t. Why? Um… Because. Obviously. Rules. Assuming a movie is shot at 24 frames per second, a two-hour movie would have about 172,800 frames. Are they afraid we’re going to turn movies into giant animated GIFs?

(Note: No such restriction if you don’t pay for your video content, but download it for free from one of the less scrupulous corners of the Internet.)

More than just screenshots, I often want to remove DRM from iTunes video for reasons that have nothing to do with piracy. Some of those reasons might include wanting to watch the video I’ve paid for on:

  • an Android phone or tablet

  • a Roku or Amazon Fire TV or other non-Apple TV device.

  • a Mac mini connected to a monitor that doesn’t support HDCP.

It was the last one that really annoyed me: I bought video from Apple’s store and played it back on a Mac, but because the monitor didn’t meet some requirement I didn’t even know existed, I couldn’t watch the video that I had purchased.

Another interesting aside: when I tried to play 1080p video on my MacBook Air, I was told it couldn’t play it back. In fact, the computer refused to even try. But after I removed the DRM, I could play the same video via VLC just fine. Curious, isn’t it?

Three ways to watch iTunes video you’ve paid for with as much flexibility as you’d get if you had downloaded it for free.

  1. Requiem - this was a Windows and Mac tool which would quickly and remove DRM from iTunes video. Unfortunately, it only works with iTunes 10 (not 11) and does not work at all with some computer hardware. However, if you have an older Mac that will only run Lion and iTunes 10, you might want to give Requiem a try first. It is the fastest of the three options, and the only free one.

  2. NoteBurner M4V Converter Plus is not free (US$50) but works very well at removing DRM from iTunes video. However, it does not support subtitles or closed captioning from iTunes video.

  3. On the other hand, m4vgear does support subtitles. It costs US$50 also, but a 30%-off coupon can usually be found by searching for m4vgear 30% coupon. Note that you may have to follow the link from the coupon site in order for the coupon to be honored. $35 isn’t a bad deal, especially if you have a lot of money invested in iTunes media.

Note that sometimes VLC will not play the subtitles from a file from m4vgear. In that case you will have to watch the movie via QuickTime (or, presumably, iTunes) to get the subtitles to play.

Both the NoteBurner and m4vgear websites look fairly… well… let’s just say generic, and it would be understandable if someone wondered if these apps were scams. I purchased the NoteBurner software and found it to be quite excellent, but when I realized that it did not preserve subtitles, I emailed support, and they: a) suggested I try m4vgear and b) gave me a full refund, even though it had been quite a long time after I purchased the app. That’s fairly excellent customer service.

If there are any TV/Movie executives reading this and have become angry that I am telling someone how to “defeat” DRM, please remember: I am a paying customer.

I will gladly show you my iTunes receipts showing that I have bought and paid for my movies and TV shows. I’ll gladly share pictures showing the VHS, DVD, and Blu-Ray discs that I have purchased.

I go to the movies, I buy movies, I watch TV shows, I buy TV shows on DVD/Blu-Ray.

I’m the kind of customer you want. Being able to watch iTunes video on my Roku or on my Android tablet makes me more likely to keep buying them.

In fact, I’m even willing to pay for software to help me do this.

At the end of the day, I’ve paid you so I can watch these movies/shows. Does it really matter to you if I’m watching them on my Roku or my Apple TV?

Port your Google Voice number to your Verizon iPhone

Recorded on April 16th, 2014

I wrote up the full story and posted it on TUAW: How to port your Google Voice number to your iPhone.