Introduction

Target Audience

• Teams needing automated RSS information aggregation
• Developers wanting to implement DingTalk group automation
• DevOps engineers interested in workflow automation
• Tech enthusiasts looking for open-source automation tools

I. Private Deployment of n8n

1.1 Basic Environment Setup

Before installing n8n, we need to create the necessary directory structure. The following commands will create the required folders in the user’s home directory:

# Switch to root user
su
# Enter password
# Create n8n main directory and subdirectories
mkdir -p /root/data/docker_data/n8n/

# Command explanation:
# mkdir: command to create directory
# -p: create parent directories if they don't exist
# /root/data/docker_data/n8n/: create n8n folder in /root/data/docker_data/ directory

# Enter n8n directory
cd /root/data/docker_data/n8n/

1.2 Docker Compose Configuration

In this step, we’ll configure Docker Compose to manage n8n and its dependencies. This approach is easier to manage and maintain compared to running containers directly.

Create docker-compose.yml file:

cd /root/data/docker_data/n8n/
vim docker-compose.yml

version: "3.8"

volumes:
  db_storage:
  n8n_storage:

services:
  n8n-postgres:
    image: postgres:16
    container_name: n8n-postgres
    restart: always
    environment:
      - POSTGRES_USER
      - POSTGRES_PASSWORD
      - POSTGRES_DB
      - POSTGRES_NON_ROOT_USER
      - POSTGRES_NON_ROOT_PASSWORD
    volumes:
      - db_storage:/var/lib/postgresql/data
      - ./init-data.sh:/docker-entrypoint-initdb.d/init-data.sh
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}']
      interval: 5s
      timeout: 5s
      retries: 10

  n8n:
    build:
      context: .
      dockerfile: Dockerfile
    image: n8n-custom
    container_name: n8n
    restart: always
    environment:
      - N8N_HOST=${N8N_HOST}
      - NODE_ENV=production
      - N8N_EDITOR_BASE_URL=${N8N_EDITOR_BASE_URL}
      - VUE_APP_URL_BASE_API=${N8N_EDITOR_BASE_URL}
      - WEBHOOK_URL=${N8N_EDITOR_BASE_URL}
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=n8n-postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
      - DB_POSTGRESDB_USER=${POSTGRES_NON_ROOT_USER}
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_NON_ROOT_PASSWORD}
      - TZ=Asia/Shanghai
      - N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=false
      - NODE_FUNCTION_ALLOW_BUILTIN=*
      - NODE_TLS_REJECT_UNAUTHORIZED=0
    ports:
      - 5678:5678
    links:
      - n8n-postgres
    volumes:
      - n8n_storage:/home/node/.n8n
    depends_on:
      n8n-postgres:
        condition: service_healthy

1.3 Customized Dockerfile Configuration

To extend n8n’s functionality, we need to add Python environment and related dependencies. This enables n8n to handle more complex data transformation tasks.

cd /root/data/docker_data/n8n/
vim Dockerfile

Create Dockerfile file, add custom Python library support:

FROM docker.n8n.io/n8nio/n8n:latest

USER root

# Install Python environment
RUN apk add --no-cache python3 py3-pip

# Create virtual environment
RUN python3 -m venv /opt/venv

# Activate virtual environment and install dependencies
ENV PATH="/opt/venv/bin:$PATH"
RUN pip install --no-cache-dir html2text beautifulsoup4

# Switch back to node user
USER node

1.4 Create .env File

cd /root/data/docker_data/n8n/
vim .env

Create .env file, add the following content:

N8N_HOST=your-server-ip
N8N_EDITOR_BASE_URL=http://your-server-ip:5678

POSTGRES_USER=admin
POSTGRES_PASSWORD=123456
POSTGRES_DB=n8n

POSTGRES_NON_ROOT_USER=admin
POSTGRES_NON_ROOT_PASSWORD=123456

1.5 Start Service

Execute the following command to start n8n. If Docker is not installed, please install Docker first:

docker compose up -d

Access http://your-ip:5678 to complete account password initialization.

II. RSS to DingTalk Bot Implementation

2.1 DingTalk Bot Setup

Before configuring the workflow, we need to complete the DingTalk bot setup. This step is crucial for subsequent message pushing.

1. In DingTalk group - Settings - Robot - Add Robot - Custom Robot
2. Select “Sign” in security settings
3. Save access_token and secret

2.2 Workflow Node Configuration

Workflow is a core concept in n8n, connecting various function nodes visually to achieve data flow and processing.

Create a new workflow, search and select nodes on the right:

• RSS Trigger
• Code
• HTTP Request

The workflow includes the following nodes:

2.3 Configure RSS Trigger

1. Select RSS Trigger node
2. Set polling interval to 1 hour
3. Choose RSS source, using Telegram channel daily news as example

https://app.yhy.gd.cn/telegram/channel/NEWSPJAPK

4. Click Fetch Test Event to test node response data

2.4 Select Code Node, Use JavaScript for DingTalk Signing

DingTalk signature generation code:

// Set signature parameters
const secret = "your_secret";
const timestamp = Math.floor(Date.now());
const stringToSign = `${timestamp}\n${secret}`;

// Use Crypto module to generate HMAC-SHA256 signature
const hmac = crypto
    .createHmac('sha256', secret) // Specify algorithm and key
    .update(stringToSign)        // Update string to be signed
    .digest('base64');           // Generate Base64 format signature

// Return signature and timestamp
return {
    timestamp,
    sign: hmac,
    content: $node["RSS"].json['contentSnippet'].replace('#', '# ').replace("\nSource: Xinhua, People's Daily, etc.", "").replaceAll("\n",'\n- ')
  };

2.5 Select HTTP Request Node, Send DingTalk Message

1. Choose Post request
2. Configure Webhook URL, select Expression in top right, which will replace content in {{}} with actual values

https://oapi.dingtalk.com/robot/send?access_token=your_access_token&timestamp={{$node["Code"].json["timestamp"]}}&sign={{$node["Code"].json["sign"]}}

3. Configure request headers

Content-Type: application/json

4. Configure request body

{
    "msgtype": "text",
    "text": {
        "content": "{{$node["HTML to Markdown"].json["content"]}}"
    }
}

5. After configuration, test the workflow

6. For unknown variable values, you can run previous nodes in n8n and drag values to corresponding positions

2.6 Click Save button in top right to save workflow, click Active button to activate workflow to execute according to set schedule

III. Extension Development and Troubleshooting

3.1 Feature Extension and Custom Development

n8n’s power lies in its extensibility. We can enhance its data processing capabilities through Python scripts.

3.2 HTML to Markdown Processing

1. Select Code node, first process RSS return content escaping

// Code node content
const htmlContent = $input.first().json.content

// Process content escaping
const escapedContent = JSON.stringify(htmlContent)
  .replace(/'/g, "\\'")  // Escape single quotes
  .slice(1, -1);        // Remove double quotes added by JSON.stringify

// Return escaped content
return {
  json: {
    escaped_content: escapedContent
  }
};

2. Use Execute Command node to call server-installed Python library for HTML to Markdown processing

Python code for processing HTML content:

python3 -c "
import html2text,re;
from bs4 import BeautifulSoup;

# Configure html2text
h=html2text.HTML2Text();
h.ignore_links=False;
h.body_width=0;
h.escape_snob=True;
h.unicode_snob=True;
h.mark_code=False;

# Preprocess HTML content
content='{{ $json.escaped_content }}'.replace('\\\\\"','\\'').replace('\\\"','\"');

# Use BeautifulSoup to extract and preserve original image tags
soup=BeautifulSoup(content,'html.parser');
img_tags={};
for img in soup.find_all('img'):
    if img.get('style') or img.get('width') or img.get('height'):
        img_str = str(img).replace('\"', '\\'');
        img_tags[img.get('src','')] = img_str;

# Convert to Markdown
md=h.handle(content);

# Process all escape characters
md=md.replace(r'\(',r'(').replace(r'\)',r')').replace(r'\[',r'[').replace(r'\]',r']').replace(r'\.', r'.').replace(r'\-',r'-').replace(r'\n',r'\n').replace(r'\!',r'!').replace('\"', '\\'');

# Replace markdown image syntax back to original HTML tags
for src, html_tag in img_tags.items():
    pattern=f'!\\[([^\\]]*)\\]\\({re.escape(src)}\\)';
    md=re.sub(pattern, html_tag, md);

print(md);"

3.3 Performance Optimization and Batch Processing

To improve workflow stability and efficiency, we need to process large amounts of data in batches.

1. Add Code node for batch processing of returned data

// Code node content
// split_batch node
const htmlContent = $input.first().json.stdout
const products = htmlContent.split('* * *');
const batchSize = 5; // Send 5 products per batch

// Divide products into batches
const batches = products.reduce((acc, product, index) => {
    const batchIndex = Math.floor(index / batchSize);
    if (!acc[batchIndex]) {
        acc[batchIndex] = [];
    }
    acc[batchIndex].push(product);
    return acc;
}, []);

return batches.map(batch => ({
    json: {
        content: batch.join('\n---\n')
    }
}));

2. Add Loop Over Items node to process batched data

• Connect Loop to next operation, like DingTalk signing and notification
• Reconnect DingTalk notification back to Loop Over Items node for next batch
• Can add loop success handling connected to Done node

3.4 Common Problem Solutions

1. Python Module Not Found

• First, ensure html2text library is installed in your n8n (already configured in Dockerfile)
• After configuring Dockerfile, restart n8n service

docker compose down
docker compose up -d

# Enter docker container to check installation
docker exec -it n8n /bin/sh

# Switch to node user (n8n running user)
su node

# Check Python environment path
which python3

# Try importing library
python3 -c "import html2text; print(html2text.__version__)"

• If installation failed, delete n8n container image and restart service

# View n8n container images
docker image ls
# Delete n8n container image
docker rmi custom-n8n
# Restart n8n service
docker compose up -d

2. DingTalk Signature Verification Failed

   • Check timestamp format
   • Verify secret correctness
   • Check server time zone
   • Try modifying workflow timezone (see images below)

3. Markdown Rendering Abnormal

   • Adjust html2text configuration
   • Check content encoding

3.5 Daily News DingTalk Notification Workflow

{
  "name": "Daily Brief",
  "nodes": [
    {
      "parameters": {
        "options": {

        }
      },
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 3,
      "position": [-100, 40],
      "id": "493ff425-c792-4e40-99da-129dddaa65e0",
      "name": "Loop Over Items"
    },
    {
      "parameters": {

      },
      "type": "n8n-nodes-base.noOp",
      "name": "Replace Me",
      "typeVersion": 1,
      "position": [1260, 160],
      "id": "d54910a6-a2d6-43b7-9677-e388a6f8e41e"
    },
    {
      "parameters": {
        "method": "POST",
        "url": "=https://oapi.dingtalk.com/robot/send?access_token=your_access_token&timestamp={{$node[\"Code\"].json[\"timestamp\"]}}&sign={{$node[\"Code\"].json[\"sign\"]}}",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "Content-Type",
              "value": "application/json"
            }
          ]
        },
        "sendBody": true,
        "bodyParameters": {
          "parameters": [
            {
              "name": "msgtype",
              "value": "markdown"
            },
            {
              "name": "markdown",
              "value": "={ title: \"# 【Daily Brief】\", text: '{{ $json.content }}' }"
            }
          ]
        },
        "options": {

        }
      },
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [900, -80],
      "id": "5dc96e02-5475-4d78-a086-bd7c196123f4",
      "name": "HTTP Request",
      "alwaysOutputData": true,
      "notesInFlow": false
    }
  ],
  "connections": {
    "Loop Over Items": {
      "main": [
        [
          {
            "node": "Code",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "active": true,
  "settings": {
    "executionOrder": "v1"
  },
  "tags": [
    {
      "name": "dingtalk"
    }
  ]
}

IV. References

• n8n Setup Tutorial
• n8n Official Documentation
• n8n-postgres
• DingTalk API Documentation
• RSS Subscription Tutorial
• n8n Workflow Market

V. Related Reading

• Docker Setup Tutorial
• Nginx Proxy Manager Setup Tutorial

This article was first published on Early Bird’s Blog, please indicate the source when reposting.