How Do You Get the Logs?
Collection is the part of a SIEM deployment that actually fails. What installs, what happens when it is installed everywhere or twice, and how a collector that falls behind is stopped from losing data quietly.
In short
- Agentless first. Windows estates use built-in event forwarding, so most of an estate is collected without installing anything on it.
- Safe to install everywhere. Collectors detect whether their role exists and do nothing, loudly, where it does not.
- Safe to install twice. Where a query covers the whole organisation, the server decides which collector owns it — the installer never has to get it right.
- A source that goes quiet is a finding, not a blank space on a chart.
- A runaway source cannot fill your disk. A circuit breaker drops the excess instead of taking the database down, and reports it as a standing finding rather than an alert.
- Repeated events are sent once. Where an endpoint produces the same event over and over, the sender ships one copy with the times it happened — keeping every event while removing most of them from the network.
- Endpoint monitoring is filtered on whether an event can be read, not on how much of it there is — measured against our own event store, not a template.
Deployments fail at collection, not at detection
Detection content is the part vendors talk about. Collection is the part that decides whether a deployment works — and it fails in ways that look like success.
A collector installed on a server that does not have the role it expects. A collector installed twice, quietly doubling every count. A forwarder that fell behind three weeks ago and has been silently discarding events since. In each case the console looks calm.
The person deploying this may be an experienced consultant — or an office manager following a PDF, or the partner's nephew who is good with computers. The deployment will be uniform, minimal, and done once: one package pushed everywhere, one policy linked at one place, one script run.
Anything requiring the installer to sort servers by role will be skipped, and a half-finished sort is worse than none because it looks finished. So the lazy deployment has to be the correct one.
Four ways in, agentless first
| Method | What it covers | Installs |
|---|---|---|
| Windows event forwarding | Domain controllers, servers, workstations | Nothing on the endpoint — built into Windows |
| Role collectors | Mail servers, web servers, backup servers | A scheduled script on that server |
| Syslog receiver | Firewalls, switches, appliances, Linux hosts | Nothing — point the device at the appliance |
| Connectors | Virtualisation, backup products, hardware management | Nothing — the appliance polls them |
A firm of six people has no software deployment tooling and no appetite for an agent on every machine. Windows already forwards its own events; using that gets most of an estate covered with a policy change rather than an installation project.
Most endpoint monitoring is noise, and we measured ours
Deep endpoint monitoring is the part that fills disks and slows searches. One Windows machine, watched for a week, produced 4.6 million events — about seven and a half every second, from a single desktop. Two kinds of event were almost all of it, and most of that carried nothing anyone could act on.
- Nearly 3 million came from one line of configuration, matching a Windows subsystem that records device pairing continuously. In the same week, that line delivered 322 events with any investigative value.
- A further 227,000 recorded that a name lookup happened without recording what was asked for — a fact nothing can act on, already covered properly by a different event.
The trap in trimming the noisiest thing is that the noisiest thing is sometimes the only record of something serious. Connections between machines inside the estate are the largest block left on that endpoint — and they are kept, because they are the only record this product holds of an attacker moving from one machine to the next. Removing them would have looked like good tuning and would have quietly removed a capability.
Those changes project a 76% reduction on that machine. It is arithmetic over one developer workstation's own traffic rather than a measurement of a typical office computer, and it is presented that way deliberately. Real sizing is confirmed on your estate, where the appliance reports what it is actually consuming.
Everywhere, twice, or nowhere
Every collector is designed against three failure cases, because all three happen.
Installed everywhere
A collector on a server without its role does nothing, sends nothing, costs nothing, and says so in words: “correctly did nothing — not every server is a file server”. Phrasing it that way is what prevents the support call. This is verified by running the collectors on a machine with none of the roles, not by reading the code.
Installed twice
Some queries return the whole organisation's records regardless of which server runs them. Four mail servers running the same collector ingest everything four times.
The damage lands on alert thresholds. Count-based rules fire at a quarter of their intended activity, which presents as an over-sensitive detection rather than a deployment mistake. Somebody raises the threshold, and the rule is then wrong in the other direction for every estate that deployed correctly.
So the server decides. The first collector to report an organisation-wide scope owns it; the others are told they are on standby and do nothing. If the owner stops reporting, another takes over automatically. No collector needs to know another exists, and the installer cannot get it wrong.
Collectors that read genuinely local data — web server logs, file permissions — run everywhere on purpose. The rule is one collector per thing the query actually covers, rather than one per estate.
Installed nowhere, or half
Coverage gaps are visible rather than silent. A source that stops reporting raises a finding, and the estate view distinguishes hosts that are reporting from hosts known to exist that are not.
Telling a broken cable from a broken estate
When collection breaks, most products light up every source at once. That is honest and almost useless: it points the operator at the estate, and they start checking machines one at a time when the answer is a network cable.
Chronolith separates the two by the evidence that actually distinguishes them, and says which it thinks it is. It also notices when the appliance's own hardware changes underneath it — memory or processors added, a network port moved — because those are silent on a virtual machine and leave the appliance quietly wrong about its own capacity.
A domain controller silent for ten minutes is a problem. A backup server that reports twice a day is not. Expected cadence is derived from what each source has actually been doing, so “this one stopped” means something for both.
A runaway source cannot take the appliance down
A misconfigured forwarder, a debug logging level left on, or a network storm can produce orders of magnitude more events than normal. Without a limit, that fills the disk and the whole system stops — including the parts watching for an attack.
- Each source's normal rate is learned from its own recent history.
- A sustained spike far above that rate has its excess dropped, not queued.
- The drop is never silent. It is reported as a standing finding naming the source and the number of events lost — deliberately not an alert.
- Thresholds are adjustable, because an estate whose normal rate is genuinely high should not have to fight its own protection.
It used to be an alert, one per source each time it tripped. On our own reference estate that produced 483 of 496 alerts — 97% of every alert the system had ever raised, burying the thirteen that were real detections, ransomware staging among them. A console that is almost entirely one message teaches people to clear it without reading, which costs exactly the alerts the product exists to raise.
“This source is sending more than usual” is also a condition with a duration rather than an event: it starts, it persists, and it ends. It now behaves like one — a single line that updates while it lasts and clears itself when the source settles.
Dropping events is a real cost and we do not pretend otherwise. The alternative is a database that fills and a SIEM that stops entirely — losing some events from one misbehaving source beats losing all events from every source. What makes it a decision rather than a silent loss is that the drop is reported, with the source named and the number of events counted.
Make the careless path the safe one
Whoever installs this will put the key wherever is most convenient. So the convenient place has to be safe, or the key must not be there at all.
A key in a scheduled task's arguments is readable by any local administrator and is captured by command-line process auditing — which Chronolith itself collects. That places the customer's own key, in plain text, into their own searchable event store.
A key distributed by group policy lands in a directory share readable by every authenticated user in the domain. This is exactly how stored group-policy passwords became a well-known vulnerability class.
Collectors therefore read their key from a file or the environment and never accept it as a command-line argument — there is no parameter to pass one.
Deployed agents authenticate differently again: each has its own identity and signs every request, with a short replay window, so a captured request cannot be replayed later.
Common questions
Do we have to install an agent on every machine?
No. Windows estates are collected through built-in event forwarding, which is a policy change rather than an installation. Agents and role collectors are for specific jobs — deeper endpoint telemetry, or reading something only that server holds.
What happens if the appliance is down or unreachable?
Windows event forwarding buffers on the source and resumes when the appliance returns, so a short outage does not lose data. A long one can exceed the source's own buffer, which is a Windows limit rather than ours — and the gap is visible afterwards rather than silently absent.
How much bandwidth and storage does this need?
Volume depends far more on what you enable than on estate size, and it is not spread evenly — a handful of very chatty event types dominate everything else. That is why the endpoint configuration is filtered on whether an event carries anything readable, which on a measured machine removed roughly three quarters of what it was sending. Sizing guidance ships with the deployment documentation, and the appliance reports its own consumption and headroom so the estimate is corrected by reality rather than trusted.
Can we send logs to Chronolith and to something else?
Yes. Windows forwarding and syslog both support multiple destinations, and Chronolith is a collector rather than an interceptor — nothing about it prevents another system receiving the same data.
Does it need internet access?
Not for collection, storage, search or reporting. Optional features reach outward — threat intelligence feeds and licence checks — and the product functions without them, degrading in stated ways rather than silently.