Files
paperlessmanager/docs/superpowers/plans/2026-05-23-agrarmonitor-polling.md
T
bjoernpoettker 1d11d8a3bd
Build and Push Multi-Platform Images / build-and-push (push) Successful in 57s
docs: add Agrarmonitor polling design plans
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 15:09:53 +02:00

1359 lines
47 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Agrarmonitor Polling Service Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Extend the Agrarmonitor module with a polling service that checks Paperless documents against Agrarmonitor invoices and updates both systems when a booking date is found.
**Architecture:** A new `AgrarmonitorWebService` handles the HTML scraping of admin7.agrarmonitor.de (livesearch, setEingangsdatum, setLieferscheinNummer). A new `AgrarmonitorPollingService` orchestrates the polling loop (cron + manual trigger), reads tag-ID settings from the DB, and updates both Agrarmonitor and Paperless. The `Client` entity gets an `AgrarmonitorBetriebId` column so the BetriebId-to-Owner mapping is configurable instead of hardcoded.
**Tech Stack:** NestJS, TypeORM (MySQL, synchronize:true), `node-html-parser`, `@nestjs/schedule` Cron, PaperlessService (already exists), React 19 + Ant Design.
---
## File Map
| Action | File |
|--------|------|
| Modify | `paperless-backend/package.json` |
| Modify | `paperless-backend/src/database/entities/client.entity.ts` |
| **Create** | `paperless-backend/src/agrarmonitor/agrarmonitor-web.service.ts` |
| **Create** | `paperless-backend/src/agrarmonitor/agrarmonitor-polling.service.ts` |
| Modify | `paperless-backend/src/agrarmonitor/agrarmonitor.controller.ts` |
| Modify | `paperless-backend/src/agrarmonitor/agrarmonitor.module.ts` |
| Modify | `paperless-backend/src/settings/settings.controller.ts` |
| Modify | `.env.example` |
| Modify | `docker-compose.yml` |
| Modify | `paperless-frontend/src/api/settings.ts` |
| Modify | `paperless-frontend/src/api/inbox.ts` |
| Modify | `paperless-frontend/src/pages/SettingsPage.tsx` |
---
### Task 1: Install node-html-parser
**Files:**
- Modify: `paperless-backend/package.json`
- [ ] **Step 1: Install the package**
```bash
cd paperless-backend && npm install node-html-parser
```
Expected: `node-html-parser` appears in `dependencies` in `package.json`.
- [ ] **Step 2: Verify TypeScript types are available**
```bash
cd paperless-backend && node -e "const { parse } = require('node-html-parser'); console.log(typeof parse)"
```
Expected: `function`
- [ ] **Step 3: Commit**
```bash
git add paperless-backend/package.json paperless-backend/package-lock.json
git commit -m "chore: add node-html-parser for Agrarmonitor HTML scraping"
```
---
### Task 2: Extend Client entity with AgrarmonitorBetriebId
**Files:**
- Modify: `paperless-backend/src/database/entities/client.entity.ts`
- [ ] **Step 1: Add the nullable column**
Replace the full file content with:
```typescript
import { Entity, PrimaryGeneratedColumn, Column } from 'typeorm';
@Entity('clients')
export class Client {
@PrimaryGeneratedColumn()
Id!: number;
@Column({ type: 'varchar', length: 45 })
Name!: string;
@Column({ type: 'int' })
PaperlessUserId!: number;
@Column({ type: 'int', nullable: true })
AgrarmonitorBetriebId!: number | null;
}
```
- [ ] **Step 2: Verify build compiles**
```bash
cd paperless-backend && npm run build 2>&1 | tail -5
```
Expected: no TypeScript errors.
- [ ] **Step 3: Commit**
```bash
git add paperless-backend/src/database/entities/client.entity.ts
git commit -m "feat: add AgrarmonitorBetriebId to Client entity"
```
---
### Task 3: Create AgrarmonitorWebService (HTML scraping)
**Files:**
- Create: `paperless-backend/src/agrarmonitor/agrarmonitor-web.service.ts`
This service wraps the three Agrarmonitor web actions: livesearch, setEingangsdatum, setLieferscheinNummer. It calls the session-authenticated `client.http` Axios instance from `AgrarmonitorService`.
- [ ] **Step 1: Create the file**
```typescript
// paperless-backend/src/agrarmonitor/agrarmonitor-web.service.ts
import { Injectable, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { parse } from 'node-html-parser';
import { AgrarmonitorService } from './agrarmonitor.service';
export interface EingangsrechnungEntry {
eingangId: number;
belegNummer: string;
interneBelegNummer: string;
kundenId: number;
betriebId: number;
buchungsDatum: Date | null;
eingangsDatum: Date | null;
}
@Injectable()
export class AgrarmonitorWebService {
private readonly logger = new Logger(AgrarmonitorWebService.name);
private readonly baseUrl: string;
constructor(
private readonly agrarmonitorService: AgrarmonitorService,
private readonly configService: ConfigService,
) {
this.baseUrl = this.configService.get<string>(
'AGRARMONITOR_BASE_URL',
'https://admin7.agrarmonitor.de',
);
}
async eingangsrechnungenLivesearch(suchstring: string): Promise<EingangsrechnungEntry[]> {
const client = await this.agrarmonitorService.getClient();
await client.http.get('/');
const searchUrl =
`/module/dateien/livesearch.php?suchstring=${encodeURIComponent(suchstring)}` +
`&stammdatum_typ=-1&mobil=-1&sensibel=-1&firma=0&itemsperpage=100000&seite=1`;
const { data: html } = await client.http.get<string>(searchUrl, { responseType: 'text' });
const root = parse('<doc>' + html + '</doc>');
const table = root.querySelector('table#dateien');
if (!table) return [];
const rows = table.querySelectorAll('tbody tr');
const results: EingangsrechnungEntry[] = [];
for (const row of rows) {
const tds = row.querySelectorAll('td');
if (tds.length < 4) continue;
if (!tds[3].text.trim().startsWith('Eingangsrechnungen')) continue;
const linkEl = tds[3].querySelector('a');
if (!linkEl) continue;
const href = linkEl.getAttribute('href') ?? '';
const eingangId = parseInt(href.split('/').pop() ?? '0', 10);
if (!eingangId) continue;
const belegText = linkEl.text;
const belegParts = belegText.split(',');
const belegNummer = belegParts[0]?.trim() ?? '';
const { data: editHtml } = await client.http.get<string>(
`/module/eingangsrechnungen/api/eingangsrechnungen.php?id=edit&rechnungId=${eingangId}`,
{ responseType: 'text' },
);
const editRoot = parse(editHtml);
const interneBelegNummer =
editRoot.querySelector('input[name="lieferscheinnummer"]')?.getAttribute('value') ?? '';
const kundenId = parseInt(
editRoot
.querySelector('select[name="rgempf"] option[selected]')
?.getAttribute('value') ?? '0',
10,
);
const betriebId = parseInt(
editRoot
.querySelector('select[name="firma_id"] option[selected]')
?.getAttribute('value') ?? '0',
10,
);
const { data: detailHtml } = await client.http.get<string>(
`/eingangsrechnungen/detail/${eingangId}`,
{ responseType: 'text' },
);
const detailRoot = parse(detailHtml);
const receivedEl = detailRoot.getElementById('receivedStatus');
let eingangsDatum: Date | null = null;
let buchungsDatum: Date | null = null;
if (receivedEl) {
const eingangsText = receivedEl.text.trim();
if (eingangsText !== 'Nicht empfangen' && eingangsText.length > 13) {
eingangsDatum = this.parseGermanDate(eingangsText.substring(13));
}
const parentText = receivedEl.parentNode?.text ?? '';
const dashIdx = parentText.lastIndexOf('-');
const buchenText = dashIdx >= 0 ? parentText.substring(dashIdx + 1).trim() : '';
if (buchenText !== 'Nicht gebucht' && buchenText.length > 11) {
buchungsDatum = this.parseGermanDate(buchenText.substring(11));
}
}
results.push({ eingangId, belegNummer, interneBelegNummer, kundenId, betriebId, buchungsDatum, eingangsDatum });
}
return results;
}
async setEingangsdatum(eingangId: number, _belegNummer: string, datum: Date): Promise<boolean> {
const client = await this.agrarmonitorService.getClient();
const params = new URLSearchParams();
params.append('datum', this.formatGermanDate(datum));
params.append('receiptID', String(eingangId));
try {
const res = await client.http.post(
'/module/eingangsrechnungen/api/updateReceived.php',
params.toString(),
{
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
Referer: `${this.baseUrl}/eingangsrechnungen/detail/${eingangId}`,
Origin: this.baseUrl,
},
},
);
return res.status < 400;
} catch (err: any) {
this.logger.error(`setEingangsdatum(${eingangId}) Fehler: ${err?.message}`);
return false;
}
}
async setLieferscheinNummer(eingangId: number, lieferscheinNummer: string): Promise<boolean> {
const client = await this.agrarmonitorService.getClient();
const { data: editHtml } = await client.http.get<string>(
`/module/eingangsrechnungen/api/eingangsrechnungen.php?id=edit&rechnungId=${eingangId}`,
{ responseType: 'text' },
);
const editRoot = parse(editHtml);
const rechnungsnummer =
editRoot.querySelector('input[name="rechnungsnummer"]')?.getAttribute('value') ?? '';
const rechnungsdatum =
editRoot.querySelector('input[name="rechnungsdatum"]')?.getAttribute('value') ?? '';
const rgempf =
editRoot
.querySelector('select[name="rgempf"] option[selected]')
?.getAttribute('value') ?? '';
const addressName =
editRoot.querySelector('input[name="addressName"]')?.getAttribute('value') ?? '';
const params = new URLSearchParams();
params.append('lieferscheinnummer', lieferscheinNummer);
params.append('rechnungsnummer', rechnungsnummer);
params.append('rechnungsdatum', rechnungsdatum);
params.append('rgempf', rgempf);
params.append('adresstext', addressName);
try {
const res = await client.http.post(
`/module/eingangsrechnungen/api/eingangsrechnungen.php?id=update&rechnungId=${eingangId}`,
params.toString(),
{
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
Referer: `${this.baseUrl}/eingangsrechnungen/detail/${eingangId}`,
Origin: this.baseUrl,
},
},
);
return res.status < 400;
} catch (err: any) {
this.logger.error(`setLieferscheinNummer(${eingangId}) Fehler: ${err?.message}`);
return false;
}
}
private parseGermanDate(str: string): Date | null {
const parts = str.trim().split('.');
if (parts.length !== 3) return null;
const [dd, mm, yy] = parts;
const year = parseInt(yy, 10) < 50 ? 2000 + parseInt(yy, 10) : 1900 + parseInt(yy, 10);
const d = new Date(year, parseInt(mm, 10) - 1, parseInt(dd, 10));
return isNaN(d.getTime()) ? null : d;
}
private formatGermanDate(date: Date): string {
const dd = String(date.getDate()).padStart(2, '0');
const mm = String(date.getMonth() + 1).padStart(2, '0');
const yy = String(date.getFullYear()).slice(-2);
return `${dd}.${mm}.${yy}`;
}
}
```
- [ ] **Step 2: Verify build compiles**
```bash
cd paperless-backend && npm run build 2>&1 | tail -5
```
Expected: no TypeScript errors.
- [ ] **Step 3: Commit**
```bash
git add paperless-backend/src/agrarmonitor/agrarmonitor-web.service.ts
git commit -m "feat: add AgrarmonitorWebService with livesearch and date setters"
```
---
### Task 4: Create AgrarmonitorPollingService
**Files:**
- Create: `paperless-backend/src/agrarmonitor/agrarmonitor-polling.service.ts`
This service:
- Seeds the two tag-ID settings on startup (`agrarmonitor_tag_fertig` default `'4'`, `agrarmonitor_tag_verbucht` default `'9'`)
- Runs on cron when `AGRARMONITOR_POLLING_CRON` env var is set
- Exposes `runPolling()` and `getPollingConfig()`/`updatePollingConfig()` for the controller
Custom field IDs (hardcoded to match the C# reference, change if your Paperless installation differs):
- `interneBelegnummer` = custom field `7`
- `Eingangsdatum` = custom field `9`
- [ ] **Step 1: Create the file**
```typescript
// paperless-backend/src/agrarmonitor/agrarmonitor-polling.service.ts
import { Injectable, Logger, OnModuleInit } from '@nestjs/common';
import { Cron } from '@nestjs/schedule';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { AgrarmonitorService } from './agrarmonitor.service';
import { AgrarmonitorWebService } from './agrarmonitor-web.service';
import { PaperlessService } from '../paperless/paperless.service';
import { Setting } from '../database/entities/setting.entity';
import { Client } from '../database/entities/client.entity';
const INTERN_BELEGNUMMER_FIELD_ID = 7;
const EINGANGSDATUM_FIELD_ID = 9;
export interface PollingResult {
processed: number;
updated: number;
skipped: number;
errors: string[];
}
@Injectable()
export class AgrarmonitorPollingService implements OnModuleInit {
private readonly logger = new Logger(AgrarmonitorPollingService.name);
constructor(
private readonly agrarmonitorService: AgrarmonitorService,
private readonly webService: AgrarmonitorWebService,
private readonly paperlessService: PaperlessService,
@InjectRepository(Setting) private readonly settingRepo: Repository<Setting>,
@InjectRepository(Client) private readonly clientRepo: Repository<Client>,
) {}
async onModuleInit() {
await this.upsertSetting('agrarmonitor_tag_fertig', '4');
await this.upsertSetting('agrarmonitor_tag_verbucht', '9');
}
@Cron(process.env['AGRARMONITOR_POLLING_CRON'] || '0 */30 * * * *')
async scheduledPolling() {
if (!process.env['AGRARMONITOR_POLLING_CRON']) return;
this.runPolling().catch((err) => this.logger.error('Cron-Polling-Fehler:', err));
}
async getPollingConfig(): Promise<{ tagFertig: string; tagVerbucht: string }> {
const [fertig, verbucht] = await Promise.all([
this.settingRepo.findOneBy({ Tag: 'agrarmonitor_tag_fertig' }),
this.settingRepo.findOneBy({ Tag: 'agrarmonitor_tag_verbucht' }),
]);
return {
tagFertig: fertig?.Wert ?? '4',
tagVerbucht: verbucht?.Wert ?? '9',
};
}
async updatePollingConfig(tagFertig: string, tagVerbucht: string): Promise<{ tagFertig: string; tagVerbucht: string }> {
await this.settingRepo.update({ Tag: 'agrarmonitor_tag_fertig' }, { Wert: tagFertig });
await this.settingRepo.update({ Tag: 'agrarmonitor_tag_verbucht' }, { Wert: tagVerbucht });
return { tagFertig, tagVerbucht };
}
async runPolling(): Promise<PollingResult> {
const result: PollingResult = { processed: 0, updated: 0, skipped: 0, errors: [] };
this.logger.log('Starte Agrarmonitor-Polling');
const [tagFertigSetting, tagVerbuchtSetting] = await Promise.all([
this.settingRepo.findOneBy({ Tag: 'agrarmonitor_tag_fertig' }),
this.settingRepo.findOneBy({ Tag: 'agrarmonitor_tag_verbucht' }),
]);
const tagFertigId = parseInt(tagFertigSetting?.Wert ?? '4', 10);
const tagVerbuchtId = parseInt(tagVerbuchtSetting?.Wert ?? '9', 10);
let amClient: Awaited<ReturnType<typeof this.agrarmonitorService.getClient>>;
try {
amClient = await this.agrarmonitorService.getClient();
} catch (err: any) {
const msg = `Connector-Fehler: ${err?.message ?? 'unbekannt'}`;
this.logger.error(msg);
return { ...result, errors: [msg] };
}
let customers: any[];
try {
customers = await amClient.fetchCustomers();
} catch (err: any) {
const msg = `Kunden-Abruf fehlgeschlagen: ${err?.message ?? 'unbekannt'}`;
this.logger.error(msg);
return { ...result, errors: [msg] };
}
for (const customer of customers.filter(
(c: any) => Number(c['ist_lieferant']) === 1 && Number(c['ist_aktiv']) === 1,
)) {
const lieferantennummer = (customer['lieferantennummer'] as string) ?? '';
const searchName = `(${lieferantennummer})`;
const displayName = this.buildCustomerName(customer, lieferantennummer);
const existing = await this.paperlessService.getCorrespondentByName(searchName);
if (!existing) {
await this.paperlessService.addCorrespondent({
name: displayName,
match: '',
matching_algorithm: 0,
is_insensitive: true,
owner: null,
});
}
}
const docsResponse = await this.paperlessService.getDocuments({
page: 1,
page_size: 9999,
truncate_content: true,
tags__id__all: tagFertigId,
});
const docs: any[] = docsResponse?.results ?? [];
this.logger.log(`${docs.length} Dokumente fertig in Agrarmonitor`);
for (const doc of docs) {
result.processed++;
const interneBelegnummer = ((doc.custom_fields as any[]) ?? []).find(
(cf: any) => cf.field === INTERN_BELEGNUMMER_FIELD_ID,
)?.value as string ?? '';
if (!interneBelegnummer) {
this.logger.log(`Dokument ${doc.id as number} hat keine interne Belegnummer`);
result.skipped++;
await this.delay(500);
continue;
}
let amResults: Awaited<ReturnType<typeof this.webService.eingangsrechnungenLivesearch>>;
try {
amResults = await this.webService.eingangsrechnungenLivesearch(interneBelegnummer);
} catch (err: any) {
this.logger.error(`Livesearch ${interneBelegnummer}: ${err?.message}`);
result.errors.push(`${interneBelegnummer}: Livesearch-Fehler`);
await this.delay(500);
continue;
}
if (amResults.length === 0) {
this.logger.log(`${interneBelegnummer} nicht in Agrarmonitor gefunden`);
result.skipped++;
await this.delay(500);
continue;
}
if (amResults.length > 1) {
this.logger.error(`${interneBelegnummer} mehrfach in Agrarmonitor gefunden`);
result.errors.push(`${interneBelegnummer}: Mehrfach gefunden`);
await this.delay(500);
continue;
}
const amDoc = amResults[0];
if (!amDoc.interneBelegNummer && interneBelegnummer) {
await this.webService.setLieferscheinNummer(amDoc.eingangId, interneBelegnummer);
}
if (!amDoc.eingangsDatum) {
const eingangsdatumField = ((doc.custom_fields as any[]) ?? []).find(
(cf: any) => cf.field === EINGANGSDATUM_FIELD_ID,
);
if (eingangsdatumField?.value) {
const eingangsdatum = new Date(eingangsdatumField.value as string);
if (!isNaN(eingangsdatum.getTime())) {
await this.webService.setEingangsdatum(amDoc.eingangId, amDoc.belegNummer, eingangsdatum);
this.logger.log(`Eingangsdatum für ${interneBelegnummer} gesetzt`);
}
}
} else if (amDoc.buchungsDatum) {
try {
let correspondentId: number | undefined;
const customer = customers.find((c: any) => Number(c.id) === amDoc.kundenId);
if (customer) {
const lieferantennummer = (customer['lieferantennummer'] as string) ?? '';
const searchName = `(${lieferantennummer})`;
const displayName = this.buildCustomerName(customer, lieferantennummer);
let corr = await this.paperlessService.getCorrespondentByName(searchName);
if (!corr) {
corr = await this.paperlessService.addCorrespondent({
name: displayName,
match: '',
matching_algorithm: 0,
is_insensitive: true,
owner: null,
});
}
if (corr) correspondentId = corr.id as number;
}
let ownerId: number | undefined;
const matchedClient = await this.clientRepo.findOneBy({
AgrarmonitorBetriebId: amDoc.betriebId,
});
if (matchedClient) ownerId = matchedClient.PaperlessUserId;
const currentTags: number[] = (doc.tags as number[]) ?? [];
const newTags = currentTags.filter((t) => t !== tagFertigId).concat([tagVerbuchtId]);
const updateData: Record<string, any> = { tags: newTags };
if (correspondentId !== undefined) updateData.correspondent = correspondentId;
if (ownerId !== undefined) updateData.owner = ownerId;
await this.paperlessService.updateDocument(doc.id as number, updateData);
this.logger.log(`Beleg ${interneBelegnummer} gebucht`);
result.updated++;
} catch (err: any) {
this.logger.error(`Update ${interneBelegnummer}: ${err?.message}`);
result.errors.push(`${interneBelegnummer}: Update-Fehler`);
}
} else {
result.skipped++;
}
await this.delay(500);
}
this.logger.log(
`Polling abgeschlossen: ${result.processed} verarbeitet, ${result.updated} aktualisiert, ${result.skipped} übersprungen, ${result.errors.length} Fehler`,
);
return result;
}
private buildCustomerName(customer: any, nummer: string): string {
const firma = (customer.firma as string) ?? '';
const nachname = (customer['nachname'] as string) ?? '';
const vorname = (customer['vorname'] as string) ?? '';
const name = firma || (nachname + (vorname ? ', ' + vorname : ''));
return `${name} (${nummer})`;
}
private delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
private async upsertSetting(tag: string, defaultValue: string): Promise<void> {
const existing = await this.settingRepo.findOneBy({ Tag: tag });
if (!existing) {
await this.settingRepo.save(
this.settingRepo.create({ Typ: 1, Wert: defaultValue, Tag: tag }),
);
}
}
}
```
- [ ] **Step 2: Verify build compiles**
```bash
cd paperless-backend && npm run build 2>&1 | tail -5
```
Expected: no TypeScript errors.
- [ ] **Step 3: Commit**
```bash
git add paperless-backend/src/agrarmonitor/agrarmonitor-polling.service.ts
git commit -m "feat: add AgrarmonitorPollingService with cron and runPolling"
```
---
### Task 5: Extend AgrarmonitorController with 3 new endpoints
**Files:**
- Modify: `paperless-backend/src/agrarmonitor/agrarmonitor.controller.ts`
Replace the entire file with:
- [ ] **Step 1: Update the controller**
```typescript
// paperless-backend/src/agrarmonitor/agrarmonitor.controller.ts
import { Body, Controller, Get, HttpCode, Post, Put } from '@nestjs/common';
import { AgrarmonitorService } from './agrarmonitor.service';
import { AgrarmonitorPollingService } from './agrarmonitor-polling.service';
import { RequirePermissions } from '../auth/permissions.decorator';
import { Permission } from '../auth/permissions.enum';
@Controller('api/agrarmonitor')
export class AgrarmonitorController {
constructor(
private readonly service: AgrarmonitorService,
private readonly pollingService: AgrarmonitorPollingService,
) {}
@Get('status')
@RequirePermissions(Permission.MANAGE_SETTINGS)
async getStatus() {
return this.service.getStatus();
}
@Post('register')
@HttpCode(200)
@RequirePermissions(Permission.MANAGE_SETTINGS)
async registerDevice(@Body() body: { pcName: string; agrarmonitorId: string }) {
return this.service.registerDevice(body.pcName, body.agrarmonitorId);
}
@Get('polling-config')
@RequirePermissions(Permission.MANAGE_SETTINGS)
async getPollingConfig() {
return this.pollingService.getPollingConfig();
}
@Put('polling-config')
@RequirePermissions(Permission.MANAGE_SETTINGS)
async updatePollingConfig(@Body() body: { tagFertig: string; tagVerbucht: string }) {
return this.pollingService.updatePollingConfig(body.tagFertig, body.tagVerbucht);
}
@Post('run-polling')
@HttpCode(200)
@RequirePermissions(Permission.MANAGE_SETTINGS)
async runPolling() {
return this.pollingService.runPolling();
}
}
```
- [ ] **Step 2: Verify build compiles**
```bash
cd paperless-backend && npm run build 2>&1 | tail -5
```
Expected: no TypeScript errors.
- [ ] **Step 3: Commit**
```bash
git add paperless-backend/src/agrarmonitor/agrarmonitor.controller.ts
git commit -m "feat: add polling-config and run-polling endpoints to AgrarmonitorController"
```
---
### Task 6: Update AgrarmonitorModule
**Files:**
- Modify: `paperless-backend/src/agrarmonitor/agrarmonitor.module.ts`
- [ ] **Step 1: Replace the module file**
```typescript
// paperless-backend/src/agrarmonitor/agrarmonitor.module.ts
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { AgrarmonitorService } from './agrarmonitor.service';
import { AgrarmonitorWebService } from './agrarmonitor-web.service';
import { AgrarmonitorPollingService } from './agrarmonitor-polling.service';
import { AgrarmonitorController } from './agrarmonitor.controller';
import { Setting } from '../database/entities/setting.entity';
import { Client } from '../database/entities/client.entity';
import { PaperlessModule } from '../paperless/paperless.module';
@Module({
imports: [
TypeOrmModule.forFeature([Setting, Client]),
PaperlessModule,
],
providers: [AgrarmonitorService, AgrarmonitorWebService, AgrarmonitorPollingService],
controllers: [AgrarmonitorController],
exports: [AgrarmonitorService],
})
export class AgrarmonitorModule {}
```
- [ ] **Step 2: Verify build compiles**
```bash
cd paperless-backend && npm run build 2>&1 | tail -5
```
Expected: no TypeScript errors.
- [ ] **Step 3: Commit**
```bash
git add paperless-backend/src/agrarmonitor/agrarmonitor.module.ts
git commit -m "feat: wire AgrarmonitorModule with TypeOrm, PaperlessModule, and new services"
```
---
### Task 7: Add GET/PUT clients endpoints to SettingsController
**Files:**
- Modify: `paperless-backend/src/settings/settings.controller.ts`
Add two methods at the end of the `// === Benutzer-Betrieb Zuordnung ===` section (around line 295), before the `// === Allgemeine Einstellungen ===` section.
- [ ] **Step 1: Add the two client endpoints**
In `paperless-backend/src/settings/settings.controller.ts`, find the line:
```typescript
// === Allgemeine Einstellungen ===
```
Add the following two methods immediately before it:
```typescript
// === Betriebe (Clients) ===
@Get('clients')
async getAllClients() {
return this.clientRepo.find({ order: { Id: 'ASC' } });
}
@Put('clients/:id')
async updateClient(@Param('id') id: string, @Body() body: { AgrarmonitorBetriebId: number | null }) {
await this.clientRepo.update(parseInt(id, 10), {
AgrarmonitorBetriebId: body.AgrarmonitorBetriebId ?? null,
});
return this.clientRepo.findOneByOrFail({ Id: parseInt(id, 10) });
}
```
- [ ] **Step 2: Verify build compiles**
```bash
cd paperless-backend && npm run build 2>&1 | tail -5
```
Expected: no TypeScript errors.
- [ ] **Step 3: Commit**
```bash
git add paperless-backend/src/settings/settings.controller.ts
git commit -m "feat: add GET/PUT /api/settings/clients endpoints for AgrarmonitorBetriebId management"
```
---
### Task 8: Update .env.example and docker-compose.yml
**Files:**
- Modify: `.env.example`
- Modify: `docker-compose.yml`
- [ ] **Step 1: Add AGRARMONITOR_POLLING_CRON to .env.example**
Find the line `AGRARMONITOR_ENCRYPTION_KEY= # optional, 16+ Zeichen für Cookie-Verschlüsselung` and add the following line immediately after it:
```
AGRARMONITOR_POLLING_CRON=0 */30 * * * * # alle 30 Minuten (leer lassen = deaktiviert)
```
- [ ] **Step 2: Add AGRARMONITOR_POLLING_CRON to docker-compose.yml**
Find the line `- AGRARMONITOR_ENCRYPTION_KEY=${AGRARMONITOR_ENCRYPTION_KEY:-}` and add the following line immediately after it:
```yaml
- AGRARMONITOR_POLLING_CRON=${AGRARMONITOR_POLLING_CRON:-}
```
- [ ] **Step 3: Commit**
```bash
git add .env.example docker-compose.yml
git commit -m "chore: add AGRARMONITOR_POLLING_CRON env var to .env.example and docker-compose.yml"
```
---
### Task 9: Frontend — Update API files
**Files:**
- Modify: `paperless-frontend/src/api/settings.ts`
- Modify: `paperless-frontend/src/api/inbox.ts`
- [ ] **Step 1: Update Client interface in inbox.ts**
In `paperless-frontend/src/api/inbox.ts`, find:
```typescript
export interface Client {
Id: number;
Name: string;
PaperlessUserId: number;
}
```
Replace with:
```typescript
export interface Client {
Id: number;
Name: string;
PaperlessUserId: number;
AgrarmonitorBetriebId: number | null;
}
```
- [ ] **Step 2: Add polling types and API methods to settings.ts**
In `paperless-frontend/src/api/settings.ts`, find the `export const agrarmonitorApi = {` block and replace it with:
```typescript
export interface PollingConfig {
tagFertig: string;
tagVerbucht: string;
}
export interface PollingResult {
processed: number;
updated: number;
skipped: number;
errors: string[];
}
export interface AgrarmonitorClient {
Id: number;
Name: string;
PaperlessUserId: number;
AgrarmonitorBetriebId: number | null;
}
export const agrarmonitorApi = {
getStatus: () =>
api.get<AgrarmonitorStatusData>('/api/agrarmonitor/status').then((r) => r.data),
registerDevice: (pcName: string, agrarmonitorId: string) =>
api
.post<{ success: boolean; message: string }>('/api/agrarmonitor/register', { pcName, agrarmonitorId })
.then((r) => r.data),
getPollingConfig: () =>
api.get<PollingConfig>('/api/agrarmonitor/polling-config').then((r) => r.data),
updatePollingConfig: (config: PollingConfig) =>
api.put<PollingConfig>('/api/agrarmonitor/polling-config', config).then((r) => r.data),
runPolling: () =>
api.post<PollingResult>('/api/agrarmonitor/run-polling', {}).then((r) => r.data),
getAllClients: () =>
api.get<AgrarmonitorClient[]>('/api/settings/clients').then((r) => r.data),
updateClient: (id: number, data: { AgrarmonitorBetriebId: number | null }) =>
api.put<AgrarmonitorClient>(`/api/settings/clients/${id}`, data).then((r) => r.data),
};
```
- [ ] **Step 3: Verify frontend TypeScript compiles**
```bash
cd paperless-frontend && npm run build 2>&1 | tail -10
```
Expected: no TypeScript errors.
- [ ] **Step 4: Commit**
```bash
git add paperless-frontend/src/api/settings.ts paperless-frontend/src/api/inbox.ts
git commit -m "feat: add polling config/run API and AgrarmonitorBetriebId to frontend API layer"
```
---
### Task 10: Frontend — Extend SettingsPage.tsx
**Files:**
- Modify: `paperless-frontend/src/pages/SettingsPage.tsx`
Two sections to update: `AgrarmonitorTab` (add polling config + run button) and `UserClientsTab` (add Betriebe management table).
#### Part A — AgrarmonitorTab
- [ ] **Step 1: Add imports**
Find the existing imports from `'../api/settings'` (around line 16). Add the new types:
```typescript
agrarmonitorApi, type AgrarmonitorStatusData,
type PollingConfig, type PollingResult, type AgrarmonitorClient,
```
(Replace the current `agrarmonitorApi, type AgrarmonitorStatusData,` with the above.)
- [ ] **Step 2: Replace the AgrarmonitorTab function**
Find the comment `// ═══════════════════════════════════════════════════════════════════` directly above `function AgrarmonitorTab()` and replace the entire `AgrarmonitorTab` function (up to the closing `}` before the next `// ═══...` comment) with:
```typescript
// ═══════════════════════════════════════════════════════════════════
// Agrarmonitor Tab
// ═══════════════════════════════════════════════════════════════════
function AgrarmonitorTab() {
const [form] = Form.useForm();
const [loading, setLoading] = useState(false);
const [registering, setRegistering] = useState(false);
const [status, setStatus] = useState<AgrarmonitorStatusData | null>(null);
const [registerResult, setRegisterResult] = useState<{ success: boolean; message: string } | null>(null);
const [pollingConfig, setPollingConfig] = useState<PollingConfig>({ tagFertig: '4', tagVerbucht: '9' });
const [savingConfig, setSavingConfig] = useState(false);
const [runningPolling, setRunningPolling] = useState(false);
const [pollingResult, setPollingResult] = useState<PollingResult | null>(null);
useEffect(() => {
agrarmonitorApi.getPollingConfig().then(setPollingConfig).catch(() => {});
}, []);
const handleLoadStatus = async () => {
setLoading(true);
setRegisterResult(null);
try {
const data = await agrarmonitorApi.getStatus();
setStatus(data);
} catch (err: any) {
const msg = err?.code === 'ECONNABORTED'
? 'Timeout Backend antwortet nicht rechtzeitig'
: (err?.response?.data?.message ?? err?.message ?? 'Netzwerkfehler');
setStatus({ connected: false, registriert: null, freigeschaltet: null, error: msg });
} finally {
setLoading(false);
}
};
const handleRegister = async () => {
const values = await form.validateFields();
setRegistering(true);
setRegisterResult(null);
try {
const result = await agrarmonitorApi.registerDevice(values.pcName, values.agrarmonitorId);
setRegisterResult(result);
if (result.success) {
message.success('Gerät erfolgreich registriert');
await handleLoadStatus();
}
} catch {
setRegisterResult({ success: false, message: 'Registrierung fehlgeschlagen' });
} finally {
setRegistering(false);
}
};
const handleSaveConfig = async () => {
setSavingConfig(true);
try {
await agrarmonitorApi.updatePollingConfig(pollingConfig);
message.success('Konfiguration gespeichert');
} catch {
message.error('Fehler beim Speichern');
} finally {
setSavingConfig(false);
}
};
const handleRunPolling = async () => {
setRunningPolling(true);
setPollingResult(null);
try {
const result = await agrarmonitorApi.runPolling();
setPollingResult(result);
} catch (err: any) {
message.error(err?.response?.data?.message ?? 'Polling fehlgeschlagen');
} finally {
setRunningPolling(false);
}
};
const renderStatusTag = (value: boolean | null, labelTrue: string, labelFalse: string) => {
if (value === null) return <Tag></Tag>;
return value
? <Tag color="success">{labelTrue}</Tag>
: <Tag color="error">{labelFalse}</Tag>;
};
return (
<div style={{ maxWidth: 600 }}>
<Typography.Title level={4}>Agrarmonitor</Typography.Title>
<Typography.Paragraph type="secondary">
Verbindungsstatus und Geräte-Registrierung für die Agrarmonitor-Schnittstelle.
Zugangsdaten werden in der <code>.env</code> konfiguriert.
</Typography.Paragraph>
<Space direction="vertical" style={{ width: '100%' }} size="large">
<div>
<Button loading={loading} onClick={handleLoadStatus}>
Status abrufen
</Button>
</div>
{status && (
<Card size="small" title="Status">
<Space direction="vertical">
<div>
<strong>Verbindung: </strong>
{status.connected
? <Tag color="success">Verbunden</Tag>
: <Tag color="error">Nicht verbunden</Tag>}
</div>
<div>
<strong>Registriert: </strong>
{renderStatusTag(status.registriert, 'Ja', 'Nein')}
</div>
<div>
<strong>Freigeschaltet: </strong>
{renderStatusTag(status.freigeschaltet, 'Ja', 'Nein')}
</div>
{status.error && (
<div style={{ color: '#ff4d4f' }}>{status.error}</div>
)}
</Space>
</Card>
)}
{status?.registriert === false && (
<Card size="small" title="Gerät registrieren">
<Form form={form} layout="vertical">
<Form.Item
name="pcName"
label="PC-Name"
rules={[{ required: true, message: 'Bitte PC-Name eingeben' }]}
>
<Input placeholder="BUERO-PC-01" />
</Form.Item>
<Form.Item
name="agrarmonitorId"
label="Agrarmonitor-ID / Firma"
rules={[{ required: true, message: 'Bitte Agrarmonitor-ID eingeben' }]}
>
<Input placeholder="Agrarmonitor-ID" />
</Form.Item>
<Button type="primary" loading={registering} onClick={handleRegister}>
Gerät registrieren
</Button>
</Form>
{registerResult && (
<div style={{ marginTop: 12 }}>
<Tag color={registerResult.success ? 'success' : 'error'}>
{registerResult.message}
</Tag>
</div>
)}
</Card>
)}
<Card size="small" title="Polling-Konfiguration">
<Space direction="vertical" style={{ width: '100%' }}>
<Row gutter={16}>
<Col span={12}>
<div style={{ marginBottom: 4 }}>Tag: Fertig in Agrarmonitor</div>
<InputNumber
value={parseInt(pollingConfig.tagFertig, 10)}
min={1}
onChange={(v) => setPollingConfig((c) => ({ ...c, tagFertig: String(v ?? 4) }))}
style={{ width: '100%' }}
/>
</Col>
<Col span={12}>
<div style={{ marginBottom: 4 }}>Tag: Verbucht</div>
<InputNumber
value={parseInt(pollingConfig.tagVerbucht, 10)}
min={1}
onChange={(v) => setPollingConfig((c) => ({ ...c, tagVerbucht: String(v ?? 9) }))}
style={{ width: '100%' }}
/>
</Col>
</Row>
<Button loading={savingConfig} onClick={handleSaveConfig}>
Konfiguration speichern
</Button>
</Space>
</Card>
<Card size="small" title="Polling ausführen">
<Space direction="vertical" style={{ width: '100%' }}>
<Button type="primary" loading={runningPolling} onClick={handleRunPolling}>
Jetzt ausführen
</Button>
{pollingResult && (
<Space wrap>
<Tag color="blue">{pollingResult.processed} verarbeitet</Tag>
<Tag color="success">{pollingResult.updated} aktualisiert</Tag>
<Tag>{pollingResult.skipped} übersprungen</Tag>
{pollingResult.errors.length > 0 && (
<Tag color="error">{pollingResult.errors.length} Fehler</Tag>
)}
</Space>
)}
{pollingResult?.errors && pollingResult.errors.length > 0 && (
<div style={{ fontSize: 12, color: '#ff4d4f' }}>
{pollingResult.errors.map((e, i) => <div key={i}>{e}</div>)}
</div>
)}
</Space>
</Card>
</Space>
</div>
);
}
```
#### Part B — UserClientsTab: add Betriebe management
- [ ] **Step 3: Add agrarmonitorApi import to UserClientsTab usage area**
The `agrarmonitorApi` and `AgrarmonitorClient` are already imported at the top of the file (added in Task 9, Step 1).
- [ ] **Step 4: Replace the UserClientsTab function**
Find the comment `// ═══════════════════════════════════════════════════════════════════` directly above `function UserClientsTab()` and replace the entire `UserClientsTab` function with:
```typescript
// ═══════════════════════════════════════════════════════════════════
// Benutzer & Betriebe Tab
// ═══════════════════════════════════════════════════════════════════
function UserClientsTab() {
const [data, setData] = useState<SettingUserClient[]>([]);
const [clients, setClients] = useState<Client[]>([]);
const [loading, setLoading] = useState(true);
const [modalOpen, setModalOpen] = useState(false);
const [form] = Form.useForm();
const [allClients, setAllClients] = useState<AgrarmonitorClient[]>([]);
const [editValues, setEditValues] = useState<Record<number, number | null>>({});
const [savingClients, setSavingClients] = useState<Record<number, boolean>>({});
const load = useCallback(async () => {
setLoading(true);
try {
const [ucs, cls, agrarClients] = await Promise.all([
settingsApi.getUserClients(),
clientsApi.getMyClients(),
agrarmonitorApi.getAllClients(),
]);
setData(ucs);
setClients(cls);
setAllClients(agrarClients);
const initialEdit: Record<number, number | null> = {};
agrarClients.forEach((c) => { initialEdit[c.Id] = c.AgrarmonitorBetriebId; });
setEditValues(initialEdit);
} finally { setLoading(false); }
}, []);
useEffect(() => { load(); }, [load]);
const handleAdd = async () => {
const values = await form.validateFields();
await settingsApi.createUserClient(values);
message.success('Zuordnung erstellt');
setModalOpen(false);
form.resetFields();
load();
};
const handleDelete = async (id: number) => {
await settingsApi.deleteUserClient(id);
message.success('Gelöscht');
load();
};
const handleSaveClient = async (clientId: number) => {
setSavingClients((s) => ({ ...s, [clientId]: true }));
try {
await agrarmonitorApi.updateClient(clientId, { AgrarmonitorBetriebId: editValues[clientId] ?? null });
message.success('Gespeichert');
} catch {
message.error('Fehler beim Speichern');
} finally {
setSavingClients((s) => ({ ...s, [clientId]: false }));
}
};
const columns: ColumnsType<SettingUserClient> = [
{ title: 'User ID', dataIndex: 'UserId', key: 'userId' },
{
title: 'Betrieb',
key: 'client',
render: (_, r) => clients.find(c => c.Id === r.ClientId)?.Name ?? r.ClientId,
},
{
title: 'Rolle',
dataIndex: 'Role',
key: 'role',
render: (r: string) => <Tag color={r === 'admin' ? 'red' : r === 'editor' ? 'blue' : 'default'}>{r}</Tag>,
},
{
title: '',
key: 'actions',
width: 80,
render: (_, record) => (
<Popconfirm title="Wirklich löschen?" onConfirm={() => handleDelete(record.Id)}>
<Button danger icon={<DeleteOutlined />} size="small" />
</Popconfirm>
),
},
];
const clientColumns: ColumnsType<AgrarmonitorClient> = [
{ title: 'Betrieb', dataIndex: 'Name', key: 'name' },
{
title: 'Agrarmonitor-BetriebId',
key: 'betriebId',
width: 220,
render: (_, record) => (
<InputNumber
value={editValues[record.Id] ?? undefined}
min={1}
placeholder=""
onChange={(v) => setEditValues((prev) => ({ ...prev, [record.Id]: v ?? null }))}
style={{ width: 120 }}
/>
),
},
{
title: '',
key: 'save',
width: 100,
render: (_, record) => (
<Button
size="small"
loading={savingClients[record.Id]}
onClick={() => handleSaveClient(record.Id)}
>
Speichern
</Button>
),
},
];
return (
<>
<Typography.Title level={5} style={{ marginTop: 0 }}>Benutzer-Betrieb-Zuordnung</Typography.Title>
<Button type="primary" icon={<PlusOutlined />} onClick={() => setModalOpen(true)} style={{ marginBottom: 16 }}>
Zuordnung hinzufügen
</Button>
<Table dataSource={data} columns={columns} loading={loading} rowKey="Id" size="small" pagination={false} />
<Divider />
<Typography.Title level={5}>Betriebe (Agrarmonitor-Mapping)</Typography.Title>
<Table
dataSource={allClients}
columns={clientColumns}
loading={loading}
rowKey="Id"
size="small"
pagination={false}
/>
<Modal title="Neue Zuordnung" open={modalOpen} onOk={handleAdd} onCancel={() => setModalOpen(false)}>
<Form form={form} layout="vertical">
<Form.Item name="UserId" label="User ID (Authentik)" rules={[{ required: true }]}>
<Input />
</Form.Item>
<Form.Item name="ClientId" label="Betrieb" rules={[{ required: true }]}>
<Select>
{clients.map(c => <Select.Option key={c.Id} value={c.Id}>{c.Name}</Select.Option>)}
</Select>
</Form.Item>
<Form.Item name="Role" label="Rolle" initialValue="editor">
<Select>
<Select.Option value="viewer">Viewer</Select.Option>
<Select.Option value="editor">Editor</Select.Option>
<Select.Option value="admin">Admin</Select.Option>
</Select>
</Form.Item>
</Form>
</Modal>
</>
);
}
```
- [ ] **Step 5: Verify frontend TypeScript compiles**
```bash
cd paperless-frontend && npm run build 2>&1 | tail -10
```
Expected: no TypeScript errors.
- [ ] **Step 6: Commit**
```bash
git add paperless-frontend/src/pages/SettingsPage.tsx
git commit -m "feat: extend AgrarmonitorTab with polling config/run and UserClientsTab with BetriebId mapping"
```
---
## Self-Review Checklist
**Spec coverage:**
- [x] `Client.AgrarmonitorBetriebId` → Task 2
- [x] Setting seed `agrarmonitor_tag_fertig` / `agrarmonitor_tag_verbucht` → Task 4 `onModuleInit`
- [x] `AgrarmonitorPollingService.runPolling()` with cron + manual trigger → Tasks 4 + 5
- [x] `GET/PUT /api/agrarmonitor/polling-config` → Task 5
- [x] `POST /api/agrarmonitor/run-polling` → Task 5
- [x] Module wiring → Task 6
- [x] `GET/PUT /api/settings/clients/:id` → Task 7
- [x] `AGRARMONITOR_POLLING_CRON` env var → Task 8
- [x] Frontend polling config UI → Task 10 Part A
- [x] Frontend Betriebe BetriebId column → Task 10 Part B
- [x] `node-html-parser` install → Task 1
- [x] All HTML-scraping calls (livesearch, setEingangsdatum, setLieferscheinNummer) → Task 3
**Notes:**
- `Betrieb` (Paperless custom field 6) and `Gruppe` are NOT set on the document during polling because these values are not stored in the `Client` entity. Only `owner` (from `Client.PaperlessUserId`) is updated. This matches the spec's explicit data model — the C# hardcoded switch is replaced by the `AgrarmonitorBetriebId` mapping.
- Custom field IDs 7 (interneBelegnummer) and 9 (Eingangsdatum) are hardcoded in `AgrarmonitorPollingService` as constants `INTERN_BELEGNUMMER_FIELD_ID` and `EINGANGSDATUM_FIELD_ID`. Change these if your Paperless installation uses different IDs.
- The cron guard `if (!process.env['AGRARMONITOR_POLLING_CRON']) return;` ensures the scheduled polling is a no-op when the env var is not set, even though the `@Cron` decorator falls back to `0 */30 * * * *`.