Dr Michael Fairbank
-
Email
m.fairbank@essex.ac.uk -
Location
1NW.3.19, Colchester Campus
-
Academic support hours
Mondays 2pm-3pm. Use zoom - see CE811 Moodle for the zoom meeting details.
Profile
Qualifications
-
BSc Mathematical Physics (Nottingham University, 1994)
-
MScKnowledge Based Systems(Edinburgh University, 1995)
-
PhD Computer Science (City University London, 2014)
Research and professional activities
Research interests
Neural Networks
Adaptive Dynamic Programming + Reinforcement Learning
Optimisation
Control Theory
Financial Forecasting
AI in games
Current research
Neural-Network Learning Algorithms
I am always trying to develop new and improved learning algorithms for training neural networks. The highlight of this work is the Deep Learning in Target Space publication.
More information about this project
Algorithms for Adaptive Dynamic Programming and Reinforcement Learning
I work on algorithms for Adaptive Dynamic Programming (which is a sister-field of Reinforcement Learning), trying to develop new algorithms / prove algorithms converge/run efficiently, etc.
One of the key outputs of this work is a convergence proof for learning with a greedy policy and function approximation for Value-Gradient Learning. This is highlighted in the paper "An Equivalence Between Adaptive Dynamic Programming With a Critic and Backpropagation Through Time", which proves equivalence between a method that uses an approximated value-function (i.e. a neural network) and a pre-existing method which has the proven convergence guarantees. Hence the proven convergence guarantees of the second method transfer over the the value-function based method.
Other interesting papers on this topic which I've published include the papers "Value-gradient learning", "A Comparison of Learning Speed and Ability to Cope Without Exploration between DHP and TD(0)", and "The divergence of reinforcement learning algorithms with value-iteration and function approximation". See my publications list for details on these papers.
More information about this project
Neurocontrol applications
I am very interested in making neural networks control systems, i.e. neurocontrol.
I have applied this technique for industrial control problems. Particularly for power system controllers, to improve energy efficiency of renewable generators. Papers on this topic include "Neural-network vector controller for permanent-magnet synchronous motor drives: Simulated and hardware-validated results" and related papers on Motors and Grid-Connected Converters.
A fun neurocontrol topic is described in the paper "A Minimal “Functionally Sentient” Organism Trained With Backpropagation Through Time", by M Pisheh Var, M Fairbank, S Samothrakis
Adaptive Behavior, linked to below. This aims to show a minimal example where we can make a neural network emulate all of the external behaviours of minimal sentient organism.
More information about this project
Teaching and supervision
Current teaching responsibilities
-
Game Artificial Intelligence (CE811)
-
Physics-Based Games (CE812)
Current supervision
Previous supervision
Degree subject: Computational Finance
Degree type: Doctor of Philosophy
Awarded date: 23/12/2022
Degree subject: Intelligent Games and Game Intelligence
Degree type: Doctor of Philosophy
Awarded date: 14/2/2019
Publications
Journal articles (14)
Abdollahi, M., Yang, X., Fairbank, M. and Nasri, M., (2023). Demand Management in Time-slotted Last-mile Delivery via Dynamic Routing with Forecast Orders. European Journal of Operational Research. 309 (2), 704-718
Pisheh Var, M., Fairbank, M. and Samothrakis, S., (2023). A Minimal “Functionally Sentient” Organism Trained with Backpropagation Through Time. Adaptive Behavior. 31 (6), 531-544
Fairbank, M., Samothrakis, S. and Citi, L., (2022). Deep Learning in Target Space. Journal of Machine Learning Research. 23, 1-46
Gao, Y., Li, S., Xiao, Y., Dong, W., Fairbank, M. and Lu, B., (2022). An Iterative Optimization and Learning-based IoT System for Energy Management of Connected Buildings. IEEE Internet of Things Journal. 9 (21), 1-1
Dong, W., Li, S., Fu, X., Li, Z., Fairbank, M. and Gao, Y., (2021). Control of a Buck DC/DC Converter Using Approximate Dynamic Programming and Artificial Neural Networks. IEEE Transactions on Circuits and Systems Part 1: Regular Papers. 68 (4), 1760-1768
Li, S., Won, H., Fu, X., Fairbank, M., Wunsch, DC. and Alonso, E., (2020). Neural-Network Vector Controller for Permanent-Magnet Synchronous Motor Drives: Simulated and Hardware-Validated Results. IEEE Transactions on Cybernetics. 50 (7), 3218-3230
Alonso, E., Fairbank, M. and Mondragon, E., (2015). Back to optimality: a formal framework to express the dynamics of learning optimal behavior. Adaptive Behavior. 23 (4), 206-215
Fu, X., Li, S., Fairbank, M., Wunsch, DC. and Alonso, E., (2015). Training Recurrent Neural Networks With the Levenberg-Marquardt Algorithm for Optimal Control of a Grid-Connected Converter. IEEE Transactions on Neural Networks and Learning Systems. 26 (9), 1900-1912
Li, S., Fairbank, M., Johnson, C., Wunsch, DC., Alonso, E. and Proao, JL., (2014). Artificial Neural Networks for Control of a Grid-Connected Rectifier/Inverter Under Disturbance, Dynamic and Power Converter Switching Conditions. IEEE Transactions on Neural Networks and Learning Systems. 25 (4), 738-750
Fairbank, M., Prokhorov, D. and Alonso, E., (2014). Clipping in Neurocontrol by Adaptive Dynamic Programming. IEEE Transactions on Neural Networks and Learning Systems. 25 (10), 1909-1920
Fairbank, M., Li, S., Fu, X., Alonso, E. and Wunsch, D., (2014). An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances. Neural Networks. 49, 74-86
Fairbank, M., Alonso, E. and Prokhorov, D., (2013). An Equivalence Between Adaptive Dynamic Programming With a Critic and Backpropagation Through Time. IEEE Transactions on Neural Networks and Learning Systems. 24 (12), 2088-2100
Fairbank, M., Alonso, E. and Prokhorov, D., (2012). Simple and fast calculation of the second-order gradients for globalized dual heuristic dynamic programming in neural networks.. IEEE Transactions on Neural Networks and Learning Systems. 23 (10), 1671-1676
Fairbank, M. and Alonso, E., (2012). Efficient calculation of the Gauss-Newton approximation of the Hessian matrix in neural networks.. Neural Computation. 24 (3), 607-610
Book chapters (1)
Fairbank, M., Prokhorov, D. and Alonso, E., (2012). Approximating Optimal Control with Value Gradient Learning. In: Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. Wiley. 142- 161. 9781118104200
Conferences (22)
Fairbank, MH. and Tuson, A., A Curvature Primal Sketch Neural Network Recognition System.
Pisheh Var, M., Fairbank, M. and Samothrakis, S., (2023). Finding Eulerian tours in mazes using amemory-augmented fixed policy function
Samothrakis, S., Matran-Fernandez, A., Abdullahi, U., Fairbank, M. and Fasli, M., (2022). Grokking-like effects in counterfactual inference
Venugopal, I., Tollich, J., Fairbank, M. and Scherp, A., (2021). A Comparison of Deep-Learning Methods forAnalysing and Predicting Business Processes
Krause, A. and Fairbank, M., (2020). Baseline win rates for neural-network based trading algorithms
Volkovas, R., Fairbank, M., Woodward, JR. and Lucas, S., (2020). Practical Game Design Tool: State Explorer
Volkovas, R., Fairbank, M., Woodward, JR. and Lucas, S., (2019). Mek: Mechanics Prototyping Tool for 2D Tile-Based Turn-Based Deterministic Games
Volkovas, R., Fairbank, M., Woodward, JR. and Lucas, S., (2019). Extracting Learning Curves From Puzzle Games
Samothrakis, S., Vodopivec, T., Fairbank, M. and Fasli, M., (2017). Convolutional-Match Networks for Question Answering
Doering, J., Fairbank, M. and Markose, S., (2017). Convolutional neural networks applied to high-frequency market microstructure forecasting
Fairbank, MH., Volkovas, R. and Perez-Liebana, D., (2017). Diversity maintenance using a population of repelling random-mutation hill climbers
Samothrakis, S., Vodopivec, T., Fasli, M. and Fairbank, M., (2016). Match memory recurrent networks
Li, S., Fu, X., Alonso, E., Fairbank, M. and Wunsch, DC., (2016). Neural-network based vector control of VSCHVDC transmission systems
Li, S., Alonso, E., Fu, X., Fairbank, M., Jaithwa, I. and Wunsch, DC., (2015). Hardware Validation for Control of Three-Phase Grid-Connected Microgrids Using Artificial Neural Networks
Li, S., Fu, X., Jaithwa, I., Alonso, E., Fairbank, M. and C. Wunsch, D., (2015). Control of Three-Phase Grid-Connected Microgrids using Artificial Neural Networks
Li, S., Fairbank, M., Fu, X., Wunsch, DC. and Alonso, E., (2013). Nested-loop neural network vector control of permanent magnet synchronous motors
Alonso, E. and Fairbank, M., (2013). Emergent and Adaptive Systems of Systems
Alonso, E., Fairbank, M. and Mondragón, E., (2012). Conditioning for least action
Li, S., Wunsch, DC., Fairbank, M. and Alonso, E., (2012). Vector control of a grid-connected rectifier/inverter using an artificial neural network
Fairbank, M. and Alonso, E., (2012). The divergence of reinforcement learning algorithms with value-iteration and function approximation
Fairbank, M. and Alonso, E., (2012). A comparison of learning speed and ability to cope without exploration between DHP and TD(0)
Fairbank, M. and Alonso, E., (2012). Value-gradient learning
Reports and Papers (1)
Fairbank, M., Samothrakis, S. and Citi, L., (2021). Deep Learning in Target Space
Grants and funding
2019
Spark EV KTP application
Innovate UK (formerly Technology Strategy Board)
2017
67% The embedding of machine learning and principles of AI technology to deploy a data-driven growth strategy in a sector leading business with a vision to disrupt the insurance industry.
Technology STrategy Board
33% The embedding of machine learning and principles of AI technology to deploy a data-driven growth strategy in a sector leading business with a vision to disrupt the insurance industry.
Hood Group Ltd
67% - Embedding intelligent systems within an UAV thermographic solar energy inspection platform to reduce UAV weight, performance and flight time
Technology STrategy Board
33% - Embedding intelligent systems within an UAV thermographic solar energy inspection platform to reduce UAV weight, performance and flight time
Above Surveying Ltd
Embedding a Machine Learning capability into the Hood Group Ltd platform.
Innovate UK (formerly Technology Strategy Board)
Embedding a Machine Learning capability into the Hood Group Ltd platform.
Hoodgroup Ltd
Improved in-pen access free pig weighing
University of Essex
Improved real time detection of wind-turbine failures - Dicam Technologies
University of Essex
Create new methods of capturing insight from current and future Preqin datasets by embedding AI and Machine Learning techniques across the unique Preqin investor platform.
Prequin
Create new methods of capturing insight from current and future Preqin datasets by embedding AI and Machine Learning techniques across the unique Preqin investor platform.
Prequin
Improved in-pen access free pig weighing
University of Essex
Improved real time detection of wind-turbine failures - Dicam Technologies
University of Essex
2016
Machine Learning for EV Range Calculation
Cab4one Limited
Contact
Academic support hours:
Mondays 2pm-3pm. Use zoom - see CE811 Moodle for the zoom meeting details.
