onfocus

The Amazon Feed Generator is still humming away on my server. Now with categories!

Say you want to stay up with the latest cat calendars that Amazon offers. It was kind of a pain before. Now you can select Books then the Calendar category within books and that specificity makes all the difference! You'll find things like the Cats in Art 2018 Wall Calendar before anyone else.



I'm not sure who these people are buying 2018 cat calendars already but I guess it doesn't hurt to be prepared.

Related, node.js is still neat.

Today I fired up a new version of the Amazon Feed Generator. It creates an RSS feed from Amazon search results so you can subscribe to them. That way you'll be alerted to new products that show up in those results.

This feed generator is one of the tools that used to live at this site. I put the original together around the time I wrote Amazon Hacks. The feed generator eventually became unnecessary because during the RSS boom of the mid-to-late 2000s, Amazon embraced feeds and offered them for many of their pages. These days they've scaled the feeds they offer way back and I still think it's handy to able to subscribe to them.

The feeds this tool generates are very simple. Each feed entry has the product name, a big image, and the new (non-used) price for the item. For example, there probably aren't too many new Bob Dylan albums coming out on vinyl, but why not subscribe to this feed just to be sure. There are a few more examples on the feed generator page.

Amazon's API has changed quite a bit over the years. My server development environment changed. Let's face it, we've all changed. So to get this running again I built it anew. I used node.js and a couple of existing packages: amazon-product-api and rss. (Those rely on other packages, which use others, and it's packages all the way down.) Thanks for sharing your code everyone! I put the code for this on Github, so you too can run your own feed generator and tweak it if you want.

There are probably bugs and I have a to-do list for things I want to clean up. So let me know if you spot something off. I'm happy another little piece of onfocus infrastructure is up and running again.

My fun with Node.js continues. Yesterday I put together a script to resize jpegs on-the-fly: imageSize.

As part of my c2bk program I've been revisiting this site and cleaning up pieces that have broken over the years. One of those pieces is a few dozen galleries of photos I posted here between 1999-2005. Before photo-sharing sites existed you had to do it yourself. The photos are small and not very good, but they are part of my history.

I added thumbnails for each gallery to my archive page. The images are small, but they're not thumbnail-sized which made that archive page inefficient to load. Picking those images out one by one and creating thumbnails seemed like a hassle. Even writing a script to do it seemed like it wasn't worth the time involved.

Then I thought, what if I could just specify the image size in the URL and have them automatically be the correct size? That sounded like a job for node.js and CloudFront. Resizing the image was quick work with sharp. It took a bit to get the URL path-parsing working correctly and to add some caching headers to the response, but then it was all set.

By putting this behind CloudFront, the images are generated on-demand when needed but should be served from Amazon's servers most of the time. Might be overkill for some thumbnails on my archive page, but it's a handy thing to have in my toolbox for future blog/photography experimentation.

The other day I was sitting around thinking about scraping a website (as one does) and I was frustrated by the complicated regex I was going to have to write to pick the data I wanted out of HTML. (Your classic two-problems problem.) So I started daydreaming about being able to scrape sites using jQuery selectors to get the elements I needed.

Then I thought, how about using a pure JavaScript environment like Node.js? I like servers and I like JavaScript, so it seems like I should like Node.js just fine. With a bit of searching I came across a great tutorial: Scraping the Web with Node.js and I was off and scraping. The cheerio module had already solved my jQuery selection problem.

At this point I could easily grab the data, but I was frustrated thinking about wrangling it into a consumable format. I was starting to get in the Node.js mindset and thought, "I bet there's a module that can help with this." A few minutes of searching later, node-rss was installed.

It took a bit of wrangling to get my first RSS feed going, but I was surprised at how quickly it all came together. It's not so different from firing up cpan and including modules in Perl scripts. But npm is just friendlier. Being able to run npm install in a new environment takes some the tedium out of your day.

So yeah, the Node.js script I wrote goes out and fetches Likes (favs, stars, hearts, etc.) from Medium, Twitter, GitHub, and Hacker News and returns RSS feeds with those likes. This lets me bring all of my favorites from those places into one spot where I can then do other things with them, hooray! I put the code up on GitHub: Recommended RSS.

Then with the help of forever I set it up as a service and I'm aggregating away. My to-do list for this little app includes caching and handling dates. I wonder if there are modules for those.