I’m a retired Unix sysadmin. Over the years I’ve built things in COBOL, FORTAN, C, perl, rexx, PHP, visual basic, various Unix shells and maybe others. Nothing has been a real “application” - mostly just utilities to help me get things done.

Now that I’m retired, and it’s cold outside, I’m curious to try some more coding - and I have an idea.

The music communities here seem to post links to YouTube. I generally use Lemmy on my phone but don’t use YouTube, or listen to music, on my phone if I can help it. I’d like to scrape a music community here and add the songs posted to a playlist in my musicbrainz account.

Does that sound like a reasonable learner project? Any suggestions for language and libraries appreciated. My preferred IDE is vim on bash and I have a home server running Linux where this could run as a daemon, or be scheduled.

  • some_guy@lemmy.sdf.org
    31·
    11 months ago

    I built two scrapers for a website that hosts images and videos using bash.

    They’re educational, I swear! /s

    I looked through the html and figured out regexes for their media. The scripts will parse all the links on the thumbnail pages and then load the corresponding primary pages with curl. On those pages, it then uses wget to grab the file. Some additional pattern matching names the file to the name of the post.

    It’s probably convoluted, but you can accomplish a lot in bash if you want to.