Many home automation solutions provide a central dashboard that serves as a nerve center for the Internet of Things (IoT) environment. From the dashboard, you can monitor and manage light levels, thermostat temperatures, and all the other settings the system controls. Some users don’t realize that the dashboard can also provide information from beyond your home network. In other words, you can use the tools of the home automation environment to fetch information from the Internet and display it in a handy at-a-glance view along with your IoT settings.
For instance, you might have personal or hobby topics that you check periodically. These personal data points might be the wave height at a local surf spot, the insect levels at a favorite camping area, or the best time of go fishing. You probably know exactly where to find this information, however, it still requires several steps to start up a web browser, click on a bookmark, and scan the page for the desired data – which might just consist of a single number or a couple of words.
In this article, I will look at how to scrape these hobby data points from web pages with just a single line of Bash code and display the results in the dashboards of two home IoT packages, Home Assistant [1] and Node-RED [2].
Getting Started
Several different automation solutions offer web-scraping tools. For example, Python has the Beautiful Soup [3] library, Home Assistant has the Scrape [4] sensor, and Node-RED has the scrape-it [5] flow. These web-scraping tools are all very usable, but unfortunately they require a detailed knowledge of the HTML/ Document Object Model for the requested page.
この記事は Linux Magazine の #271/June 2023: Smart Home 版に掲載されています。
7 日間の Magzter GOLD 無料トライアルを開始して、何千もの厳選されたプレミアム ストーリー、8,500 以上の雑誌や新聞にアクセスしてください。
すでに購読者です ? サインイン
この記事は Linux Magazine の #271/June 2023: Smart Home 版に掲載されています。
7 日間の Magzter GOLD 無料トライアルを開始して、何千もの厳選されたプレミアム ストーリー、8,500 以上の雑誌や新聞にアクセスしてください。
すでに購読者です? サインイン
URL filtering with Pi-hole Into the Funnel
Supporting browser plug-ins, network-based DNS blockers like Pi-hole help protect you against online tracking and unwanted content.
Artificial intelligence on the Raspberry Pi Learning Experience
You don't need a powerful computer system to use Al. We show what it takes to benefit from Al on the Raspberry Pi and what tasks the small computer can handle.
MakerSpace Manage your greenhouse with a Raspberry Pi Pico W Sheltered Growth
You can safely assign some greenhouse tasks to a Raspberry Pi Pico W, such as controlling ventilation, automating a heater, and opening and closing windows.
Control Center
Tipi gives you complete control of more than 100 applications and services. A mouse click is all it takes to install the apps.
In One Fell Swoop
Topgrade detects all the package managers installed on a system and executes them one by one at the command line.
Go Faster!
The fastest way through a curve on a racetrack is along the racing line. Instead of heading for Indianapolis, Mike Schilli trains his reflexes with a desktop application written in Go, just to be on the safe side.
Math Magic
MathLex lets you easily transform handwritten math formulas to digital format and use them on the web.
Custom Repair Toolkit
You can do more with System Rescue than just repair broken systems. By adding tools and scripts, you can create a custom rescue environment that meets your needs.
At Your Disposal
Debvm lets you quickly create a temporary virtual machine with a small memory footprint, ideal for testing scripts or mixing repositories
A Fresh Breeze
Vanilla OS, an immutable filesystem, seamlessly integrates applications from other distributions with an innovative container-based package manager.
