Inventing something old
I'm back from the joint application design sessions, and happy with the progress we made. But one discussion stands out as having taken longer than it should have.
We got to the point of deciding how our systems would physically talk to each other. They have some data to send to us, and we have some data for them. The usual requirements apply: the data has to be validated, the channel has to be secure, the transport has to be reliable, and the process has to be auditable. Still it took an hour to work through the details.
What we agreed upon was delivering XML files over SFTP. Standard features apply: one folder for incoming and one for outgoing, acknowledgement files, XML schema for validation, and exception reports emailed to a distribution list. In particular, we spent several minutes talking about what to do on days on which no data is exchanged. Do we send an empty file or no file at all?
This seems familiar
This problem has been solved before. In fact, this exact solution has been done before. There is nothing new here. Even so, this lengthy conversation among seasoned professionals had the air of invention. There were several problems raised, multiple solutions proposed, and many points argued. But in the end, what we "invented" together is a well-known pattern.
I don't mind that the solution we agreed upon uses older technology. Sure, web services or REST would have been more in vogue. But even with these newer standards, you have to agree upon a pattern of use. What was so frustrating is that this is a pattern, and yet the group spent an hour rediscovering it.
If you install any ESB solution, it will include components out-of-the-box that implement this pattern. You just configure them with a polling frequency, a transport protocol, a schema, and an email address. No code required.
The value of patterns is that you can work at a higher level of abstraction. All the details have been solved for you in advance. This saves time and improves communication.
And, BTW, the correct answer is to send an empty file. We finally agreed upon that, and congratulated ourselves for our brilliance.