How to automatically search Craigslist and notify you of new postings for free!
So a good friend of mine introduced me to a curious problem that I helped him solve. So there are a few bad commercial solutions to automatically search Craigslist for new postings and notify you in some fashion.
- AdRavage works well enough, however the results they email out are often several hours old by the time it sends you the email (at least for the free version).
- Ad Finder Pro which that code is horrible, and badly documented. I tried getting it to run and got various php errors, and the php dependencies are not documented. In the end all I got was a bunch of php errors and no emails. The features leave a lot to be desired, such as no searching by price or by specific category.
So after failing at these various methods I dug into the problem and found a workable solution that is free.
Craigslist has had RSS feeds for a long time, and you can pull an RSS feed of nearly any category or search you can make on Criagslist. Therefore all we need is a RSS feed reader that will notify you of changes to the RSS feed. So after a fairly short time I found RSS2Email which with the proper configuration can be easily used to accomplish this task.
- I will assume you have a Ubuntu system/server setup and running
- Install RSS2Email
sudo apt-get install rss2email
- Copy example rss2email config file
cp /usr/share/doc/rss2email/examples/config.py ~/.rss2email/
- Edit config.py file, change attributes like:
- DEFAULT_FROM
- SMTP_SEND
- {SMTP settings}
- Then run the command to create a new feed database.
r2e new {email address}
- Note you can use your carriers’ email to text function to get these messages on your phone! Only suggested if you have a unlimited texting plan!
- Now you can add a RSS feed URL to be monitored. Note the added quotes around the URL are required the & will throw off the bash command. You can get the RSS url at the bottom right of any category or search results page on Craigslist.
r2e add "{RSS URL}"
- Add the command to your crontab to run the rss2email on a regular basis
crontab -e
- Add the following to your crontab file
*/5 * * * * r2e run
Note the first time this runs on a new feed it will send you a lot of email! You can avoid this by running the following the first time you add a new URL
r2e run --no-send
Have fun, you can add as many RSS urls you want by running the add command from above. Just be careful Craigslist may ban your IP if you hit their servers too often with too many searches. I wouldn’t recommend going below a 5 minute cron interval.
This entry was posted on Sunday, May 6th, 2012 at 12:41 am and is filed under OSS, Technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.