Overview
I've decided to download my dev.to posts so that I can store them in git. That means that if the site ever goes down, I have a copy of my blog content. Step 1, is to get a list of my posts.
The dev.to API
The API is documented here.
The following endpoint will give a list of the latest articles for a given user:
https://dev.to/api/articles/latest?username=melvincarvalho
It supports pagination, each page will contain 30 articles by default. This can be increased with the per_page query parameter, which we will set to 1000. That should be enough for most users.
Replace melvincarvalho with your own username
The Code
So I wrote a JavaScript script that will pull out a list of my articles.
#!/usr/bin/env node
// requires
const argv = require('minimist')(process.argv.slice(2))
const fs = require('fs')
const path = require('path')
const $ = require('child_process').execSync
// data
globalThis.data = {
api: 'https://dev.to/api/articles/latest',
dataDir: path.join(__dirname, '..', 'posts'),
filename: 'index.json',
perPage: 1000,
user: 'melvincarvalho'
}
// init
data.api = argv.api || data.api
data.dataDir = argv.dataDir || data.dataDir
data.filename = argv.filename || data.filename
data.perPage = argv.perPage || data.perPage
data.user = argv._[0] || data.user
console.log('data', data)
// main
let postsUri = `${data.api}`
postsUri += `?per_page=${data.perPage}`
postsUri += `&username=${data.user}`
const cmd = `curl '${postsUri}'`
console.log('cmd', cmd)
const json = JSON.parse($(cmd).toString())
// output
if (!fs.existsSync(data.dataDir)) {
fs.mkdirSync(data.dataDir, { recursive: true })
}
const output = JSON.stringify(json, null, 2)
const outFile = path.join(data.dataDir, data.filename)
console.log('output', output)
fs.writeFileSync(outFile, output)
Usage: ./getposts.js [username]
The Code Explained
First we initialize the endpoint and username. Then we run some curl to get the result, and finally we format it and write it to a file.
The Output
Running this code will get a list of articles and save it to a file called posts/index.json
The JSON output can be seen here and the current script is here.
Now that I have a list of articles, it should be possible to download the markdown from individual article, too. I will hopefully cover that in a future post.
Top comments (0)