Errors during testing in OsEngine and other platforms.

When conducting robot tests, each of us assumes that the results will be accurate. However, the accuracy of the results depends on many factors. And tests can and will generate errors. Testing errors are inevitable.

You must understand that any testing has its margin of error. Furthermore, these errors can vary across different trading platforms. By developing a trading robot in OsEngine, MT4, TsLab, we risk obtaining completely different final results.

What can cause this error in tests:

• The trading logic of the robot, which generates logical nonsense.
• Idiosyncrasies in order execution.
• Data for testing.
• Methods of module operation for testing (vector/event-based).

Let's discuss this.

The trading logic of the robot, which generates logical nonsense.

Let's consider an example. Inputs:

• Testing on hourly candlesticks.
• Position opens with a BuyAtStop pending order, meaning a breakout of a certain level.
• In the robot's logic, we subscribe to the position opening event.
• In the position opening logic, we simultaneously set a stop and a take profit order.

Very simple logic. However, without understanding the testing logic and how data is fed into it, one can simply create a profitable equity curve in this way.

What actually happens in the example:

• Candlestick data will be fed into the robot once an hour.
• Data will be provided to the robot in the form of an OHLC trade tape, with four trades that will determine the execution of stop orders and take profit orders.
• In the end, within the candlestick, when there is a message that the position has opened, the robot will instantly place a take profit order and a stop order on the same candlestick. And the tester/optimizer will attempt to execute these orders within the same candlestick.
• It is possible that the take profit order will be executed more often simply because in your tester logic, the comparison of the position's take profit based on OHLC occurs before the stop.
• As a result of this logical confusion, we end up with an equity curve in the tester that goes vertically upwards. This is only due to logical errors arising from improper application of stop/take profit orders in testing the robot on data not intended for this purpose.

IMPORTANT!

In this example, when subscribing to the position opening event and setting stop/take profit orders within it, in order to avoid errors and logical mistakes, tests should be conducted ON TICK DATA. Alternatively, you should change your logic so that the robot takes action with stop loss and take profit orders only after the candlestick is closed. Simultaneously, you should place ONLY a stop loss or ONLY a take profit order.

Logical nonsense is the most frequent and dangerous misuse of the tester/optimizer that can occur. Within this misuse, without understanding the inner workings of the tester, users can simply DRAW profits for themselves.

Idiosyncrasies in order execution.

Fig. 1. Order execution settings.

Limit orders can generally be executed in the following ways:

• By crossing the order price with the instrument price.
• By touching the order price with the instrument price.
• 50-50%. The execution is first done one way and then the other.

Market orders can be executed in the following ways:

• At the close of the current candlestick.
• At the open of the next candlestick.
• At the price of the market order itself, without considering the candlestick prices.

Data for testing.

Candlesticks contain only four values. And if tests are conducted on them, certain types of orders in candlestick testing introduce a significant margin of error:

• Market orders.
• Stop loss and take profit orders placed simultaneously.

Tick data generally yield better results when using the above-mentioned scenarios, but they are also not a 100% guarantee that your orders are executed exactly where they would be in real trading. Problems with tick data:

• Execution based on trades, rather than order book, does not provide 100% accuracy in determining entry points as in real life.
• Testing large volumes of data for a year, two, or ten is not possible or close to impossible. This makes certain types of testing, such as Walk-Forwards or Cross-Tests on sufficiently deep data to understand strategy robustness, impossible.

Architecture of testing module (vector/event-based).

Vector-based testing architecture is a way of conducting tests where data is taken for the entire period, and the trading algorithm runs through this data in a loop, making certain trading actions.

Features:

• Speed. It is tens of times faster than event-based architecture.
• Looking into the future. Wealth-Lab users have written dozens of posts about this. It is a scary thing during tests on this type of architecture. Very dangerous.
• Difficulties in transitioning to live trading because the robot's logic does not describe it the same way as it will be in reality.

Event-based testing architecture is a way of conducting tests where a so-called exchange emulator is created. It tries to make the robot not understand whether it is trading in the tester or in real life, based on the available data by sequentially delivering them.

Features:

• Low speed.
• Absence of looking into the future.
• Ease of transitioning to live trading.

So what should be done?

Accept this fact as it is. Understand that there will be errors in any case. And for mass tests, you will still have to use candles, allowing for the possibility that the test results may differ from what will happen in reality with some error.

If you have any difficulties or questions, please write to the support chat. Link