Tomasz Lewandowski

My material is other people's files, and the recurring lesson is that the specification is rarely the problem. A feed that fails is usually failing on one of four things, and I check them in order. An undeclared entity stops everything. An ampersand inside text is only legal as part of a defined entity, and a bare one is a parse error at the character where it appears, so the whole file is rejected rather than the one bad field. It is the single most common reason a valid-looking feed fails to load. An undeclared namespace prefix does the same. A document using a prefix without binding it fails to parse, and the error points at the prefix rather than at the missing declaration. A mismatched encoding declaration produces a file that parses and returns wrong data, which is worse than a refusal because it looks like it worked. It is also easy to introduce by generating a file in one encoding and writing a declaration naming another. Truncation is the quiet one. A file cut off partway through is not valid, but a consumer reading incrementally may accept the records before the cut and silently ignore the rest, which is worse than an error because the failure looks like an empty update. I cover schemas for feeds because validating against one catches all four before delivery, and it is the only point where you control the outcome. The same four checks catch most of what arrives, and they take less time than the retry cycle they replace.