Category: Papers

Improving reinforcement learning algorithms: towards optimal learning rate policies

Published on arxiv:

Improving reinforcement learning algorithms: towards optimal learning rate policies

Abstract: This paper investigates to what extent we can improve reinforcement learning algorithms. Our study is split in three parts. First, our analysis shows that the classical asymptotic convergence rate O(1/\sqrt{N}) is pessimistic and can be replaced by O((log(N)/N)^{\beta}) with 1/2≤β≤1 and N the number of iterations. Second, we propose a dynamic optimal policy for the choice of the learning rate (\gamma_{k})_{k\text{\ensuremath{\ge}}0} used in stochastic algorithms. We decompose our policy into two interacting levels: the inner and the outer level. In the inner level, we present the PASS algorithm (for “PAst Sign Search”) which, based on a predefined sequence (\gamma_{k}^{o})_{k\text{\ensuremath{\ge}}0}, constructs a new sequence (\gamma_{k}^{i})_{k\text{\ensuremath{\ge}}0} whose error decreases faster. In the outer level, we propose an optimal methodology for the selection of the predefined sequence (\gamma_{k}^{o})_{k\text{\ensuremath{\ge}}0}. Third, we show empirically that our selection methodology of the learning rate outperforms significantly standard algorithms used in reinforcement learning (RL) in the three following applications: the estimation of a drift, the optimal placement of limit orders and the optimal execution of large number of shares.

From quadratic Hawkes processes to super-Heston rough volatility models with Zumbach effect

Published on arxiv:

From quadratic Hawkes processes to super-Heston rough volatility models with Zumbach effect

Abstract: Using microscopic price models based on Hawkes processes, it has been shown that under some no-arbitrage condition, the high degree of endogeneity of markets together with the phenomenon of metaorders splitting generate rough Heston-type volatility at the macroscopic scale. One additional important feature of financial dynamics, at the heart of several influential works in econophysics, is the so-called feedback or Zumbach effect. This essentially means that past trends in returns convey significant information on future volatility. A natural way to reproduce this property in microstructure modeling is to use quadratic versions of Hawkes processes. We show that after suitable rescaling, the long term limits of these processes are refined versions of rough Heston models where the volatility coefficient is enhanced compared to the square root characterizing Heston-type dynamics. Furthermore the Zumbach effect remains explicit in these limiting rough volatility models.

Optimal make take fees in a multi market maker environment

Just published on arxiv:

Optimal make take fees in a multi market maker environment

Abstract: Following the recent literature on make take fees policies, we consider an exchange wishing to set a suitable contract with several market makers in order to improve trading quality on its platform. To do so, we use a principal-agent approach, where the agents (the market makers) optimise their quotes in a Nash equilibrium fashion, providing best response to the contract proposed by the principal (the exchange). This contract aims at attracting liquidity on the platform. This is because the wealth of the exchange depends on the arrival of market orders, which is driven by the spread of market makers. We compute the optimal contract in quasi explicit form and also derive the optimal spread policies for the market makers. Several new phenomena appears in this multi market maker setting. In particular we show that it is not necessarily optimal to have a large number of market makers in the presence of a contracting scheme.

From asymptotic properties of general point processes to the ranking of financial agents

Just published on arxiv:

 

From asymptotic properties of general point processes to the ranking of financial agents by Othmane Mounjid, Mathieu Rosenbaum and Pamela Saliba

 

Abstract: We propose a general non-linear order book model that is built from the individual behaviours of the agents. Our framework encompasses Markovian and Hawkes based models. Under mild assumptions, we prove original results on the ergodicity and diffusivity of such system. Then we provide closed form formulas for various quantities of interest: stationary distribution of the best bid and ask quantities, spread, liquidity fluctuations and price volatility. These formulas are expressed in terms of individual order flows of market participants. Our approach enables us to establish a ranking methodology for the market makers with respect to the quality of their trading.

Optimal auction duration: A price formation viewpoint

Just published on arxiv:

Optimal auction duration: A price formation viewpoint by Paul Jusselin, Thibaut Mastrolia, Mathieu Rosenbaum

Abstract: We consider an auction market in which market makers fill the order book during a given time period while some other investors send market orders. We define the clearing price of the auction as the price maximizing the exchanged volume at the clearing time according to the supply and demand of each market participants. Then we derive in a semi-explicit form the error made between this clearing price and the fundamental price as a function of the auction duration. We study the impact of the behavior of market takers on this error. To do so we consider the case of naive market takers and that of rational market takers playing a Nash equilibrium to minimize their transaction costs. We compute the optimal duration of the auctions for 77 stocks traded on Euronext and compare the quality of price formation process under this optimal value to the case of a continuous limit order book. Continuous limit order books are found to be usually sub-optimal. However, in term of our metric, they only moderately impair the quality of price formation process. Order of magnitude of optimal auction durations is from 2 to 10 minutes.

The information content of high frequency traders aggressive orders: recent evidences

New paper by Pamela Saliba: The information content of high frequency traders aggressive orders: recent evidences:

 

Abstract

This empirical study uses a unique recent data set provided by the French regulator “Autorité des Marchés Financiers” and gives some evidence concerning the impact of aggressive orders on the price formation process and the information content of these orders according to the different order flow categories (high frequency traders, agency participants and proprietary participants). As expected, we find that the price impact of aggressive orders consuming exactly the quantity present at the best limit is higher than that of the ones consuming less than the quantity present at the best limit. Furthermore, the price impact is an increasing function with respect to the consumed share in percentage. We show that these price impact disparities are sustainable over time: both price impacts are permanent. On the contrary, the impact of orders consuming more than the quantity present at the best limit starts to diminish one second after the aggressive order. In contrast to previous literature, we find that the aggressive orders of HFTs are more informed than the ones of agency and proprietary members. This new finding may be an indicator of the evolution of high frequency traders activity over the years.

From Glosten-Milgrom to the whole limit order book and applications to financial regulation

New paper out at arxiv:

From Glosten-Milgrom to the whole limit order book and applications to financial regulation

Abstract:

We build an agent-based model for the order book with three types of market participants: informed trader, noise trader and competitive market makers. Using a Glosten-Milgrom like approach, we are able to deduce the whole limit order book (bid-ask spread and volume available at each price) from the interactions between the different agents. More precisely, we obtain a link between efficient price dynamic, proportion of trades due to the noise trader, traded volume, bid-ask spread and equilibrium limit order book state. With this model, we provide a relevant tool for regulators and market platforms. We show for example that it allows us to forecast consequences of a tick size change on the microstructure of an asset. It also enables us to value quantitatively the queue position of a limit order in the book.

Co-impact: Crowding effects in institutional trading activity

New paper out at arxiv:

Co-impact: Crowding effects in institutional trading activity

Abstract:

This paper is devoted to the important yet unexplored subject of crowding effects on market impact, that we call “co-impact”. Our analysis is based on a large database of metaorders by institutional investors in the U.S. equity market. We find that the market chiefly reacts to the net order flow of ongoing metaorders, without individually distinguishing them. The joint co-impact of multiple contemporaneous metaorders depends on the total number of metaorders and their mutual sign correlation. Using a simple heuristic model calibrated on data, we reproduce very well the different regimes of the empirical market impact curves as a function of volume fraction ϕ: square-root for large ϕ, linear for intermediate ϕ, and a finite intercept I(0) when ϕ→0. The value of I(0) grows with the sign correlation coefficient. Our study sheds light on an apparent paradox: How can a non-linear impact law survive in the presence of a large number of simultaneously executed metaorders?

No-arbitrage implies power-law market impact and rough volatility

New paper out at arxiv:

No-arbitrage implies power-law market impact and rough volatility

Abstract:

Market impact is the link between the volume of a (large) order and the price move during and after the execution of this order. We show that under no-arbitrage assumption, the market impact function can only be of power-law type. Furthermore, we prove that this implies that the macroscopic price is diffusive with rough volatility, with a one-to-one correspondence between the exponent of the impact function and the Hurst parameter of the volatility. Hence we simply explain the universal rough behavior of the volatility as a consequence of the no-arbitrage property. From a mathematical viewpoint, our study relies in particular on new results about hyper-rough stochastic Volterra equations.