Crawling a whole blog

[TomLucidor]

Assuming that a WordPress blog (or equivalent) does not have that much protection, what is the fastest way to do an operation similar to HTTRack (GUI full site scraper), Heritrix (InternetArchive's saver), and Grab-Site (ArchiveTeam's saver)? If the blog gets updated regularly, how does one avoid duplicate pages?
Q1: heard of Ferret, Crawlee, and AutoScraper? Seems like people are automating even more now
Q2: What do you think of Playwright and Puppeteer? Could they be used to offload scraping before article extraction with Trafilatura?

[adbar]

You'd need to use the --explore function on the command-line with --backup-dir html/ to replicate the functionality but Trafilatura would also extract the content.
In some cases it makes more sense to get the data first and then use Trafilatura locally on the downloaded content (which also answers Q2).

Q1: This is not the same, requires a lot of training, or is not as efficient (depending on the use case and the package).