Self-hosted tool for staying up-to-date with web-site content changes.
Deploy this app to Linode with a free $100 credit!
Detect website content changes and perform meaningful actions - trigger notifications via Discord, Email, Slack, Telegram, API calls and many more.
Live your data-life pro-actively.
Don't have time? Let us host it for you! try our $8.99/month subscription - use our proxies and support! , half the price of other website change monitoring services and comes with unlimited watches & checks!
Available when connected to a playwright content fetcher (included as part of our subscription service)
Fill in text boxes, click buttons and more, setup your changedetection scenario.
Using the Browser Steps configuration, add basic steps before performing change detection, such as logging into websites, adding a product to a cart, accept cookie logins, entering dates and refining searches.
After Browser Steps have been run, then visit the Visual Selector tab to refine the content you're interested in. Requires Playwright to be enabled.
Need an actual Chrome runner with Javascript support? We support fetching via WebDriver and Playwright!
POST
or GET
and other methodsWe recommend and use Bright Data global proxy services, Bright Data will match any first deposit up to $100 using our signup link.
Please star this project and help it grow! https://github.com/dgtlmoon/changedetection.io/
With Docker composer, just clone this repository and..
$ docker-compose up -d
Docker standalone
$ docker run -d --restart always -p "127.0.0.1:5000:5000" -v datastore-volume:/datastore --name changedetection.io dgtlmoon/changedetection.io
:latest
tag is our latest stable release, :dev
tag is our bleeding edge master
branch.
Alternative docker repository over at ghcr - ghcr.io/dgtlmoon/changedetection.io
See the install instructions at the wiki https://github.com/dgtlmoon/changedetection.io/wiki/Microsoft-Windows
Check out our pypi page https://pypi.org/project/changedetection.io/
$ pip3 install changedetection.io
$ changedetection.io -d /path/to/empty/data/dir -p 5000
Then visit http://127.0.0.1:5000 , You should now be able to access the UI.
Now with per-site configurable support for using a fast built in HTTP fetcher or use a Chrome based fetcher for monitoring of JavaScript websites!
docker pull dgtlmoon/changedetection.io
docker kill $(docker ps -a -f name=changedetection.io -q)
docker rm $(docker ps -a -f name=changedetection.io -q)
docker run -d --restart always -p "127.0.0.1:5000:5000" -v datastore-volume:/datastore --name changedetection.io dgtlmoon/changedetection.io
docker-compose pull && docker-compose up -d
See the wiki for more information https://github.com/dgtlmoon/changedetection.io/wiki
XPath, JSONPath, jq, and CSS support comes baked in! You can be as specific as you need, use XPath exported from various XPath element query creation tools.
(We support LXML re:test
, re:math
and re:replace
.)
ChangeDetection.io supports a massive amount of notifications (including email, office365, custom APIs, etc) when a web-page has a change detected thanks to the apprise library. Simply set one or more notification URL's in the [edit] tab of that watch.
Just some examples
discord://webhook_id/webhook_token
flock://app_token/g:channel_id
gitter://token/room
gchat://workspace/key/token
msteams://TokenA/TokenB/TokenC/
o365://TenantID:AccountEmail/ClientID/ClientSecret/TargetEmail
rocket://user:password@hostname/#Channel
mailto://user:pass@example.com?to=receivingAddress@example.com
json://someserver.com/custom-api
syslog://
And everything else in this list!
Now you can also customise your notification content and use Jinja2 templating for their title and body!
Detect changes and monitor data in JSON API's by using either JSONPath or jq to filter, parse, and restructure JSON as needed.
This will re-parse the JSON and apply formatting to the text, making it super easy to monitor and detect changes in JSON API results
For more complex parsing, filtering, and modifying of JSON data, jq is recommended due to the built-in operators and functions. Refer to the documentation for more specifc information on jq.
One big advantage of jq
is that you can use logic in your JSON filter, such as filters to only show items that have a value greater than/less than etc.
See the wiki https://github.com/dgtlmoon/changedetection.io/wiki/JSON-Selector-Filter-help for more information and examples
When you enable a json:
or jq:
filter, you can even automatically extract and parse embedded JSON inside a HTML page! Amazingly handy for sites that build content based on JSON, such as many e-commerce websites.
<html>
...
<script type="application/ld+json">
{
"@context":"http://schema.org/",
"@type":"Product",
"offers":{
"@type":"Offer",
"availability":"http://schema.org/InStock",
"price":"3949.99",
"priceCurrency":"USD",
"url":"https://www.newegg.com/p/3D5-000D-001T1"
},
"description":"Cobratype King Cobra Hero Desktop Gaming PC",
"name":"Cobratype King Cobra Hero Desktop Gaming PC",
"sku":"3D5-000D-001T1",
"itemCondition":"NewCondition"
}
</script>
json:$..price
or jq:..price
would give 3949.99
, or you can extract the whole structure (use a JSONpath test website to validate with)
The application also supports notifying you that it can follow this information automatically
See the wiki https://github.com/dgtlmoon/changedetection.io/wiki/Proxy-configuration , we also support using BrightData proxy services where possible
Raspberry Pi and linux/arm/v6 linux/arm/v7 arm64 devices are supported! See the wiki for details
Supports managing the website watch list via our API
Do you use changedetection.io to make money? does it save you time or money? Does it make your life easier? less stressful? Remember, we write this software when we should be doing actual paid work, we have to buy food and pay rent just like you.
Firstly, consider taking out a change detection monthly subscription - unlimited checks and watches , even if you don't use it, you still get the warm fuzzy feeling of helping out the project. (And who knows, you might just use it!)
Or directly donate an amount PayPal
Or BTC 1PLFN327GyUarpJd7nVe7Reqg9qHx5frNn
I offer commercial support, this software is depended on by network security, aerospace , data-science and data-journalist professionals just to name a few, please reach out at dgtlmoon@gmail.com for any enquiries, I am more than glad to work with your organisation to further the possibilities of what can be done with changedetection.io
Please login to review this project.
No reviews for this project yet.
Safe database schema change and version control for DevOps …
A one-of-a-kind resume builder that keeps your privacy in m…
Application to manage your domains and other internet asset…
Comments (0)
Please login to join the discussion on this project.