Build a Website Monitoring System On Your VPS Server

Use Case

Not everyone may need it, but I encountered a problem where I wanted to gather detailed information about my competitors’ (in the e-commerce industry) website sales and new product releases. However, I didn’t want to be glued to my computer like I would be for futures or forex trading, constantly monitoring their website.

So, I thought of using a script to do the job. The script’s functionality would be to notify me through email if a competitor sells a product, providing the name and price of the sold item. It would also notify me about newly launched products, including their names and prices. This would be helpful for analyzing my competitors’ sales volume and trends and making informed decisions about which products to sell in my own store.

Additionally, this script can be modified for various other purposes, such as monitoring the rise of your TikTok followers, tracking the progress of other events or news, or even staying updated on current hot topics.

Existing Solutions in the Market

Of course, the idea I mentioned earlier is applicable to every company or store in reality, and there are dedicated programs available for this purpose. Some good website monitoring tools include Visualping, Distill Web Monitor, and Wachete, among others. These tools are professional and well-developed, but they often come with a cost if you want to use them extensively.

There are also similar free products available, and I have tried them. However, they have limitations, such as allowing only 10 email notifications for a maximum of one website change per day. If you exceed this limit, you’ll have to pay for additional services. It’s simply not sufficient!

So, why not write your own code and use it?

The Code You Can Write Yourself

import requests
from bs4 import BeautifulSoup
import smtplib
from email.mime.text import MIMEText
import time

# Website URL
url = "https://"

# Email parameters
sender = 'your_sender_email'
receiver = 'your_receiver_email'
smtp_server = 'smtp.xxx.com'  # SMTP server address of the sender's email
smtp_port = 465  # SMTP port of the sender's email
username = 'your_sender_email'
password = 'your_sender_email_password'

def send_email(subject, body):
    # Create MIMEText email
    msg = MIMEText(body)
    msg['Subject'] = subject
    msg['From'] = sender
    msg['To'] = receiver

    # Send email
    with smtplib.SMTP_SSL(smtp_server, smtp_port) as server:
        server.login(username, password)
        server.sendmail(sender, receiver, msg.as_string())

def get_product_info(product_url):
    # Get product information
    response = requests.get(product_url)
    soup = BeautifulSoup(response.text, 'html.parser')
    title = soup.find('h1', class_='h3').text
    price = soup.find('span', class_='h2').text
    stock = soup.find('span', class_='js-product-stock').text
    return (title, price, stock)

def check_product_sold_out(product_url):
    # Check if the product is sold out
    response = requests.get(product_url)
    soup = BeautifulSoup(response.text, 'html.parser')
    if soup.find('div', class_='product-sold-out'):
        return True
    else:
        return False

# Regularly check the products
while True:
    # Get the webpage content
    response = requests.get(url)
    soup = BeautifulSoup(response.text, 'html.parser')

    # Find the product list
    product_list = soup.find('div', class_='js-product-list')

    # Check if each product is sold out
    for product in product_list.find_all('a', class_='product-card'):
        product_url = product['href']
        product_title = product.find('h2').text
        if check_product_sold_out(product_url):
            # The product is sold out, send an email notification
            subject = f'Product Sold Out: {product_title}'
            body = f'Product Name: {product_title}\n'
            send_email(subject, body)
        else:
            # The product is not sold out, check if it's a new product
            try:
                # Try to get the product information, if it fails, it means it's a new product
                product_title, product_price, product_stock = get_product_info(product_url)
            except:
                # It's a new product, send an email notification
                subject = f'New Product Released: {product_title}'
                body = f'Product Name: {product_title}\nStock: {product_stock}\nPrice: {product_price}\n'
                send_email(subject, body)

    # Wait for a certain period of time before checking again
    time.sleep(300)

Explanation: This is a monitoring program for web stores. Once there’s a new product release or a product is sold, the program will promptly notify you, just like a spy!

Here’s an additional gift

import requests
import hashlib
import time
import smtplib
from email.mime.text import MIMEText

url = 'https://'

def get_hash(url):
    response = requests.get(url)
    return hashlib.sha256(response.content).hexdigest()

def send_email(content):
    sender = ''
    receiver = ''
    password = ''
    smtp_server = ''
    smtp_port = 465

    message = MIMEText(content)
    message['From'] = sender
    message['To'] = receiver
    message['Subject'] = 'Website Change Alert'

    server = smtplib.SMTP_SSL(smtp_server, smtp_port)
    server.login(sender, password)
    server.sendmail(sender, receiver, message.as_string())
    server.quit()

current_hash = get_hash(url)
while True:
    new_hash = get_hash(url)
    if new_hash != current_hash:
        send_email('Website content has changed.')
        current_hash = new_hash
    else:
        time.sleep(30)

Explanation: This code is simple. It compares webpage hashes, and whenever there’s a change, it sends an email notification. It can be used in any scenario!

How to Use This Code

nohup python3 web_monitoring.py > output.log 2>&1 &

The code provided above is in Python, and once you make the necessary modifications, you can use it directly. Here’s a method to run it in the background:

You can set up a Python environment on your VPS and follow the instructions mentioned above to run the code automatically. It’s quite convenient, isn’t it?”