Technical interview task

Running the program

You can run the api server using the following command

$ DB_CONNECTION=/path/to/database.sqlite3 uvicorn app.api:app

Process

We validate the request, if the request does not have all attributes or invalid data (e.g. no limit price in a limit order) the request is rejected and the user gets an error
We store a new order in our persistence layer (sqlite3 in our example)
We submit the order to the exchange
We update the status of the order

Flow of an order

Developer -> API: Create order
activate API
API -> API : Validate order request

alt Invalid request
Developer <- API: Send failure response
end

API -> Database: Store order record
API -> Exchange: Place order
API <- Exchange: Receive response

alt Success
API -> Database: Mark order as placed
Developer <- API: Send success response
else Failure
API -> Database: Mark as failed
Developer <- API: Send failure response
end

deactivate API

Considerations

Persistence layer

I picked sqlite3 as a stand in for a relational database server that is easy to use for testing/development. In a real world scenario a client/server relational database could be considered, e.g. postgresql.

A relational database is a rather versatile pick, allowing a plethora of querying patterns. This versatility comes at the cost of performance. Not knowing the exact persistence layer requirements a relational database is a decent choice that should handle most load patterns in the beginning. Knowing more about the querying patterns and having a demand for more throughput, one could could consider e.g. storing relevant orders in memory and implementing a write ahead logging based system.

Exchange latency/througput

Exchange operations showcase a very high latency. That shouldn't lead anyone to think that the exchange can't handle a large number of requests, but it means that we may need to handle a large number of in-flight http requests. Ideally we are able to perform the place_order request aysnchronously, e.g. via async/await.

Backpressure considerations

Using async/await interfaces sounds like a good idea, but in reality it can lead to significant problems. Especially because async/await apis often hide the concept of back pressure. This prevents the developers of the program to fine tune the throughput of the system and setting upper boundaries (e.g. by using fixed sized queues) which would prevent system overload.

We could consider using a more explicit concurrency approach such as a basic actor model or having a thread pool executing the place order requests.

Guaranteed executions and retries

This is in response to the third (been saved in the database AND it is guaranteed that the order will be placed on the stock exchange.) and fifth requirement. (exchange should not impact the reliability of the POST /orders endpoint)

What does it mean to guarantee a placement?

We can only guarantee that an order was placed once we got the result of the exchange.

Placing an order is not just the expression of an intent to buy/sell a certain instrument at a given price. There's also a timing component to consider.

You may be willing to buy an instrument now for a given price, but not in five minutes when the prices have changed.

Therefore guaranteeing an order placement is trickier than expected. It is a time sensitive operation. While a retail client that wants to buy a stock as a long term investment may not care all that much about the exact timing of an order placement, institutional clients such as liquidity providers using the brokerage do care, they may only place an order in the hopes of it getting filled in the next e.g. 300ms and intend to cancel it afterwards.

For such a client guaranteeing the placement may not be in their best interest as they may prefer not to interact with the market at all instead of the uncertainty of when the submitted order is placed. A stale order can cost you more money than not quoting at all.

Considering this context we are going to fail fast and transparently and not e.g. store the order in the database and retrying to place the order a couple of times using a background job.

Reliability

Unless we have some kind of system to internalize order flow we are dependent on our partner exchanges to be available, otherwise we can not run our brokerage, therefore if the exchange is not available we can only respond with a predefined error code.

Improvements

Idempotency: Networks are unrealiable, it's therefore best to have some kind of idempotency key provided by the api client that can ensure that an order is only ever placed once even when multiple attempts are made.
Advanced order controls As previously mentioned, different clients have different timing requirements and some may be okay with a higher uncertainty when an order is placed due to stock exchange downtimes. By allowing clients to express their risk appetite and to prioritize placement certainty over latency by allowing placement retries we can cover both the retail as well as the institutional client's requirements.

5.1 KiB Raw Permalink Blame History