When you develop your apps locally, you probably (hopefully) use something like Xdebug. However, how do you debug production and staging errors? The answer is logs.

By sending log messages in multiple stages of a request through your application, we can see what branches of your logic the user was passed through, and what data they used to get there. The goal of the logs should primarily be to make it as easy as possible for you to recreate the user's issues locally. Knowing what data your users hit given endpoints with, you can quickly reproduce it locally.

Why care?

In a restaurant, the waiter takes your order, writes it down, and takes it over to the kitchen. They don't come back to ask you what you ordered a second time.

Our applications can be likened to restaurants. We provide a service for our customers, and minimise the friction for the customer. It means if something goes wrong, we don't ask "what did you do to cause the error". In fact, if the customer finds the problem before we do, we're already behind.

And that's why you log everything happening in your app – when something goes wrong, you're able to track down what went wrong, and how the issue was caused. If you know what data they sent to the API, at what time, from what timezone (time-related errors are the worst), you're in a much better position to debug the issue. You can simply run the application locally, submit the same data your user did, and watch the error occur again.

Beyond error tracking, logs give you statistics. If you have log messages throughout processes, you can easily find out how many people get a validation error returned for a given form, which might mean you have a frontend UX issue. You can see how many people decide to leave out optional data, how long each request takes (and filter that geographically to decide where in the world you launch your next application cluster). In short: you can infer a lot from just logs.

Where should I log?

Many times when I tell people to log more, I'm usually asked "what should I really log?", arguing that logging everything clutters your codebase. I disagree, it's not clutter – knowing how your customer was just billed is just as much a part of the application as billing them.

In short, log as much as possible, in as many places as possible. Log storage is a whole lot cheaper than a full day's worth of debugging because you lack data. In a way, logs are just like comments, except they can provide you with contextual data when things do go wrong.

If you for some reason don't want to log everywhere, at least log all incoming request data (make sure you exclude sensitive data such as passwords, social security numbers, credit card numbers, etc). This will let you reproduce failing requests as you have the same data. This is by far the greatest help for debugging errors, but other logs will let you dig in to quickly understand where the error occurred, and can also provide the added bonus of a bunch of statistics.

To conclude, adding more logs to your application can mean the difference between 30 minutes of debugging and 8 hours. Having the data available means giving off a better impression to the customer (because you don't have to ask them what they were doing to make it go wrong), and means you can reproduce the issue much faster.