title
Watch Me Build a Trading Bot

description
I've built a cryptocurrency trading bot called GradientTrader, and in this video I'll show you the tools I used to build it! It uses a graphical interface that lets you back-test on historical data, simulate paper trading, and implement a custom trading strategy for the real markets. The technique I used was a cutting edge Deep Reinforcement Learning strategy called Multi Agent Actor Critic. I'll explain it all here, enjoy! Code for this video: https://github.com/llSourcell/Watch-Me-Build-a-Trading-Bot Please Subscribe! And like. And comment. That's what keeps me going. Want more education? Connect with me here: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology instagram: https://www.instagram.com/sirajraval I also have an ongoing challenge for any Wizard brave enough to start their own startup. See details + rewards here: https://github.com/llSourcell/Build-an-AI-Startup-with-PyTorch AI in Finance: https://www.youtube.com/watch?v=7vunJlqLZok Make Money with Tensorflow 2.0: https://youtu.be/WS9Nckd2kq0 How to Make Money with Tensorflow: https://www.youtube.com/watch?v=HhqhFbwiaig&t=1s 7 Ways to Make Money with Machine Learning: https://www.youtube.com/watch?v=mrRfpiAwad0&t=117s Watch me Build an AI Startup: https://www.youtube.com/watch?v=NzmoPqte4V4&t=166s Intro to Tensorflow: https://www.youtube.com/watch?v=2FmcHiLCwTU&list=PL2-dafEMk2A7EEME489DsI468AB0wQsMV Join us in the Wizards Slack channel: http://wizards.herokuapp.com/ Hit the Join button above to sign up to become a member of my channel for access to exclusive live streams! Join us at the School of AI: https://theschool.ai/ Signup for my newsletter for exciting updates in the field of AI: https://goo.gl/FZzJ5w And please support me on Patreon: https://www.patreon.com/user?u=3191693 Join my AI community: http://chatgptschool.io/ Sign up for my AI Sports betting Bot, WagerGPT! (500 spots available): https://www.wagergpt.co

detail
{'title': 'Watch Me Build a Trading Bot', 'heatmap': [{'end': 1089.767, 'start': 958.178, 'weight': 0.702}, {'end': 2261.836, 'start': 2228.403, 'weight': 0.753}], 'summary': 'Demonstrates the development of an automated trading bot achieving 12% profit in backtesting, covers setting up automated live trades using api keys for real-time trading, creating a trading bot with deep reinforcement learning, building a trading algorithm for cryptocurrency, exploring gecko code base, reinforcement learning techniques, and developing custom trading strategies with a focus on machine learning for financial market prediction.', 'chapters': [{'end': 153.857, 'segs': [{'end': 87.254, 'src': 'embed', 'start': 0.069, 'weight': 0, 'content': [{'end': 0.91, 'text': 'hello world.', 'start': 0.069, 'duration': 0.841}, {'end': 5.213, 'text': "it's sir raj and i've built an automated trading bot in this video.", 'start': 0.91, 'duration': 4.303}, {'end': 7.515, 'text': "i'm going to show you how i built it.", 'start': 5.213, 'duration': 2.302}, {'end': 10.417, 'text': "what you're seeing behind me is my automated trading bot.", 'start': 7.515, 'duration': 2.902}, {'end': 19.264, 'text': "it's running locally on my machine, but it's a user interface that i could easily upload to google cloud aws, whatever cloud service provider i want,", 'start': 10.417, 'duration': 8.847}, {'end': 23.268, 'text': 'for both personal use or i could turn it into a service and sell it.', 'start': 19.264, 'duration': 4.004}, {'end': 28.13, 'text': "So what you're seeing here is the result of what's called backtesting.", 'start': 23.948, 'duration': 4.182}, {'end': 36.674, 'text': 'And backtesting is a strategy in financial automation that involves taking an algorithm and applying it to historical data to see how well it does.', 'start': 28.21, 'duration': 8.464}, {'end': 42.857, 'text': 'And the result for this backtesting span was about 12% in profit.', 'start': 37.255, 'duration': 5.602}, {'end': 45.479, 'text': 'So I had a starting price, an ending price.', 'start': 43.418, 'duration': 2.061}, {'end': 49.32, 'text': 'I had a time span of three months to test this on, and it made about 12% in profit.', 'start': 45.799, 'duration': 3.521}, {'end': 54.844, 'text': "So what you're seeing here is the chart of Bitcoin prices over the last three months.", 'start': 50.401, 'duration': 4.443}, {'end': 61.409, 'text': 'And all the green dots are where my algorithm decided to buy, and all the red dots are where it decided to sell.', 'start': 55.325, 'duration': 6.084}, {'end': 67.154, 'text': 'And ideally what we want are the green dots to be lower than the red dots, right? Buy low, sell high.', 'start': 61.91, 'duration': 5.244}, {'end': 68.755, 'text': 'And that was the case.', 'start': 67.854, 'duration': 0.901}, {'end': 76.421, 'text': 'I made 12% profit in this back test, and here are all my round trips, my profit and loss, the percentage, et cetera.', 'start': 68.795, 'duration': 7.626}, {'end': 79.204, 'text': 'but this is a really, really cool user interface.', 'start': 76.801, 'duration': 2.403}, {'end': 80.446, 'text': "it's called gradient trader.", 'start': 79.204, 'duration': 1.242}, {'end': 87.254, 'text': "that's what i've called it, and what you can do is you can pull data from many different exchanges.", 'start': 80.446, 'duration': 6.808}], 'summary': 'Sir raj built an automated trading bot that generated 12% profit through backtesting using historical bitcoin data.', 'duration': 87.185, 'max_score': 0.069, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o69.jpg'}, {'end': 166.437, 'src': 'embed', 'start': 130.976, 'weight': 2, 'content': [{'end': 132.057, 'text': 'Now this might take a while.', 'start': 130.976, 'duration': 1.081}, {'end': 132.637, 'text': 'Okay, great.', 'start': 132.137, 'duration': 0.5}, {'end': 135.519, 'text': 'So it had a 9% loss, not good.', 'start': 133.017, 'duration': 2.502}, {'end': 142.904, 'text': 'But the strategy that worked well for me on this particular testing set was RSI.', 'start': 135.919, 'duration': 6.985}, {'end': 147.129, 'text': "But there's gonna be several other strategies we're gonna talk about in this video.", 'start': 143.584, 'duration': 3.545}, {'end': 150.092, 'text': 'And this one made about, okay, 6%.', 'start': 147.489, 'duration': 2.603}, {'end': 151.634, 'text': "So it's always different, it's always different.", 'start': 150.092, 'duration': 1.542}, {'end': 153.857, 'text': "But that's the basic idea here.", 'start': 152.175, 'duration': 1.682}, {'end': 156.14, 'text': 'We can easily switch to live trades as well.', 'start': 153.937, 'duration': 2.203}, {'end': 157.121, 'text': 'That was back testing.', 'start': 156.16, 'duration': 0.961}, {'end': 166.437, 'text': "But for live trades, All we have to do is click on TradeBot, and that's gonna actually make trades using the APIs of whatever service we use.", 'start': 157.522, 'duration': 8.915}], 'summary': 'Test strategy rsi had 9% loss, 6% gain. other strategies to discuss. live trades via tradebot.', 'duration': 35.461, 'max_score': 130.976, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o130976.jpg'}], 'start': 0.069, 'title': 'Creating an automated trading bot and backtesting', 'summary': 'Details the development of an automated trading bot that achieved a 12% profit in backtesting by employing various strategies and historical bitcoin data from the poloniex exchange.', 'chapters': [{'end': 153.857, 'start': 0.069, 'title': 'Automated trading bot and backtesting', 'summary': 'Details the creation of an automated trading bot that achieved a 12% profit in backtesting using different strategies and historical bitcoin data from the poloniex exchange.', 'duration': 153.788, 'highlights': ['The backtesting of the automated trading bot resulted in a 12% profit over a three-month span, with the algorithm making buy and sell decisions based on historical Bitcoin prices.', 'The user interface, named Gradient Trader, allows for data pulling from various exchanges and testing different strategies, with the RSI strategy yielding a 6% profit in backtesting.', 'The automated trading bot can be easily uploaded to cloud service providers for personal use or for commercial purposes.', 'Different strategies, such as PPO and RSI, were tested on historical data, with the RSI strategy proving to be the most successful, yielding a 6% profit.', "The user interface provides visual representations of buy and sell decisions, allowing for easy analysis of the algorithm's performance."]}], 'duration': 153.788, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o69.jpg', 'highlights': ['The backtesting of the automated trading bot resulted in a 12% profit over a three-month span, with the algorithm making buy and sell decisions based on historical Bitcoin prices.', 'The user interface, named Gradient Trader, allows for data pulling from various exchanges and testing different strategies, with the RSI strategy yielding a 6% profit in backtesting.', 'Different strategies, such as PPO and RSI, were tested on historical data, with the RSI strategy proving to be the most successful, yielding a 6% profit.', 'The automated trading bot can be easily uploaded to cloud service providers for personal use or for commercial purposes.', "The user interface provides visual representations of buy and sell decisions, allowing for easy analysis of the algorithm's performance."]}, {'end': 373.903, 'segs': [{'end': 183.031, 'src': 'embed', 'start': 153.937, 'weight': 0, 'content': [{'end': 156.14, 'text': 'We can easily switch to live trades as well.', 'start': 153.937, 'duration': 2.203}, {'end': 157.121, 'text': 'That was back testing.', 'start': 156.16, 'duration': 0.961}, {'end': 166.437, 'text': "But for live trades, All we have to do is click on TradeBot, and that's gonna actually make trades using the APIs of whatever service we use.", 'start': 157.522, 'duration': 8.915}, {'end': 174.265, 'text': "All we have to do is go to Config, add an API key and then choose the exchange we wanna make trades on, whether it's Binance or Bitfinex,", 'start': 166.838, 'duration': 7.427}, {'end': 180.03, 'text': "whatever it is, add our public and private key and then it's gonna literally make trades for us in real time.", 'start': 174.265, 'duration': 5.765}, {'end': 183.031, 'text': 'Now, this is something that is an ongoing process.', 'start': 180.35, 'duration': 2.681}], 'summary': 'The system allows for easy live trading on various exchanges using api keys, enabling real-time automated trades.', 'duration': 29.094, 'max_score': 153.937, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o153937.jpg'}, {'end': 241.376, 'src': 'embed', 'start': 216.395, 'weight': 1, 'content': [{'end': 223.801, 'text': "The idea I had was let's take this um theory, this space of financial transactions, of financial market prediction,", 'start': 216.395, 'duration': 7.406}, {'end': 231.107, 'text': "and let's apply some of the most cutting edge ai technology to it to see how well we can do right, because we have that domain,", 'start': 223.801, 'duration': 7.306}, {'end': 237.172, 'text': 'knowledge of ai theory that even the huge banks like barclays and you know chase, jp, morgan,', 'start': 231.107, 'duration': 6.065}, {'end': 241.376, 'text': "you know what have you they don't have because they're not in this in the way that we are.", 'start': 237.172, 'duration': 4.204}], 'summary': 'Applying cutting edge ai to financial market prediction for outperforming big banks.', 'duration': 24.981, 'max_score': 216.395, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o216395.jpg'}, {'end': 382.248, 'src': 'embed', 'start': 353.445, 'weight': 2, 'content': [{'end': 356.268, 'text': "And there's no human involved because it's decentralized.", 'start': 353.445, 'duration': 2.823}, {'end': 361.094, 'text': "So it's very easy to access these APIs and it's also just fun.", 'start': 356.429, 'duration': 4.665}, {'end': 362.616, 'text': 'Crypto is always fun.', 'start': 361.295, 'duration': 1.321}, {'end': 369.681, 'text': "That's the stack Node Express Mongo for the database view.js, which is a newer one for me.", 'start': 364.058, 'duration': 5.623}, {'end': 373.903, 'text': 'I had to learn in a day which is for the front-end, which is really cool.', 'start': 369.681, 'duration': 4.222}, {'end': 382.248, 'text': "It's very similar to React in that it considers different components as these high-level objects that we deal with.", 'start': 373.983, 'duration': 8.265}], 'summary': 'Decentralized apis make accessing crypto fun; stack includes node, express, mongo, and view.js.', 'duration': 28.803, 'max_score': 353.445, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o353445.jpg'}], 'start': 153.937, 'title': 'Automated trading and ai prediction', 'summary': 'Covers setting up automated live trades using api keys for real-time trading on exchanges like binance and bitfinex, and building ai for financial market prediction with a focus on user interface and strategy, referencing established banks. additionally, it discusses building a cryptocurrency prediction bot using node.js, gecko trading javascript repository, convnet.js, sentiment analysis, and the bitcoin api for high frequency trading, emphasizing automation and decentralized transactions in cryptocurrency.', 'chapters': [{'end': 195.138, 'start': 153.937, 'title': 'Automated live trading setup', 'summary': 'Outlines the process of setting up automated live trades using api keys, allowing for real-time trading on exchanges like binance or bitfinex, providing a hassle-free solution for ongoing trades without manual intervention.', 'duration': 41.201, 'highlights': ['The process involves setting up API keys and choosing the exchange for real-time trades, such as Binance or Bitfinex.', 'Automated live trading allows for hassle-free ongoing trades without manual intervention, functioning like a cron job on a server.', 'Clicking on TradeBot enables the system to make trades using the APIs of the chosen service, streamlining the process for real-time trading.']}, {'end': 237.172, 'start': 196.121, 'title': 'Building ai for financial market prediction', 'summary': 'Explores building a deep reinforcement learning agent for financial market prediction using cutting edge ai technology, with a focus on user interface and strategy, and references to established banks like barclays and jp morgan.', 'duration': 41.051, 'highlights': ['Building a deep reinforcement learning agent for financial market prediction using cutting edge AI technology.', 'The importance of having a user interface for ease of use and efficiency.', 'Applying AI theory to financial market prediction to assess its effectiveness, with reference to established banks like Barclays and JP Morgan.']}, {'end': 373.903, 'start': 237.172, 'title': 'Building a cryptocurrency prediction bot', 'summary': 'Discusses building a cryptocurrency prediction bot using node.js, gecko trading javascript repository, convnet.js for reinforcement learning, sentiment analysis, and the bitcoin api for high frequency trading, emphasizing the potential for automation and decentralized transactions in cryptocurrency.', 'duration': 136.731, 'highlights': ['The stack used for building the cryptocurrency prediction bot includes Node.js, Gecko trading JavaScript repository, Convnet.js for reinforcement learning, sentiment analysis, and the Bitcoin API for high frequency trading, highlighting the focus on automation and decentralized transactions in cryptocurrency.', 'The use of Node.js for message passing, saving data to a database, and implementing algorithms on the back end allows for dynamic elements and algorithms to be written entirely in JavaScript, indicating the versatility of JavaScript in financial applications.', "The incorporation of Convnet.js, a JavaScript library for reinforcement learning, and the sentiment analysis library for evaluating news articles about a specific stock, such as Bitcoin, demonstrates the bot's ability to make data-driven trading decisions based on real-time market sentiment.", 'The emphasis on cryptocurrency for automation due to its lightning fast transactions, high frequency trading, and machine payments, as well as its decentralized nature, underlines the suitability of cryptocurrency for developing trading bots and automated financial systems.']}], 'duration': 219.966, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o153937.jpg', 'highlights': ['Automated live trading involves setting up API keys and choosing the exchange for real-time trades, such as Binance or Bitfinex.', 'Building a deep reinforcement learning agent for financial market prediction using cutting edge AI technology.', 'The stack used for building the cryptocurrency prediction bot includes Node.js, Gecko trading JavaScript repository, Convnet.js for reinforcement learning, sentiment analysis, and the Bitcoin API for high frequency trading, highlighting the focus on automation and decentralized transactions in cryptocurrency.']}, {'end': 752.112, 'segs': [{'end': 436.354, 'src': 'embed', 'start': 373.983, 'weight': 4, 'content': [{'end': 382.248, 'text': "It's very similar to React in that it considers different components as these high-level objects that we deal with.", 'start': 373.983, 'duration': 8.265}, {'end': 385.13, 'text': 'Message passing occurs between these components very easily.', 'start': 382.308, 'duration': 2.822}, {'end': 390.751, 'text': 'So the steps in this tutorial are for me to first talk about what makes a good trading bot.', 'start': 386.09, 'duration': 4.661}, {'end': 393.252, 'text': "And then we'll talk about the architecture of this code.", 'start': 391.051, 'duration': 2.201}, {'end': 397.852, 'text': 'You and I will install it together on our local machine so you can get it up and running.', 'start': 393.612, 'duration': 4.24}, {'end': 407.274, 'text': "And then we'll make our own customs trading strategy by using several technical indicators and then implementing it with deep reinforcement learning,", 'start': 398.353, 'duration': 8.921}, {'end': 410.795, 'text': 'specifically actor critic, which is a type of bleeding edge technique.', 'start': 407.274, 'duration': 3.521}, {'end': 412.215, 'text': "I'll talk about that at the end.", 'start': 411.055, 'duration': 1.16}, {'end': 418.759, 'text': "So what is a trading bot? A trading bot, it's an algorithm that makes trades for you.", 'start': 412.816, 'duration': 5.943}, {'end': 422.623, 'text': "It's a set of instructions, repeatable instructions that happens every time.", 'start': 418.799, 'duration': 3.824}, {'end': 431.13, 'text': 'And so our job as a human is to encode those instructions and try to think of all the edge cases beforehand and hopefully it works.', 'start': 422.683, 'duration': 8.447}, {'end': 436.354, 'text': "And so what's a simple example of a trading algorithm? Let's start with that.", 'start': 432.731, 'duration': 3.623}], 'summary': 'The tutorial covers creating a trading bot using react-like components, message passing, and implementing a custom trading strategy using deep reinforcement learning, specifically actor critic.', 'duration': 62.371, 'max_score': 373.983, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o373983.jpg'}, {'end': 521.964, 'src': 'embed', 'start': 494.186, 'weight': 3, 'content': [{'end': 496.548, 'text': 'The price of tomorrow is gonna be the price of yesterday.', 'start': 494.186, 'duration': 2.362}, {'end': 498.87, 'text': 'So that is called the naive strategy.', 'start': 496.848, 'duration': 2.022}, {'end': 501.031, 'text': 'And this is what the equation looks like.', 'start': 499.03, 'duration': 2.001}, {'end': 505.034, 'text': 'Y hat t plus one equals y at the previous time step.', 'start': 501.572, 'duration': 3.462}, {'end': 510.019, 'text': 'And it turns out if we do that, and here it is, Here is that strategy.', 'start': 505.535, 'duration': 4.484}, {'end': 511.26, 'text': 'This is the result.', 'start': 510.64, 'duration': 0.62}, {'end': 512.039, 'text': "Here's our prediction.", 'start': 511.32, 'duration': 0.719}, {'end': 513.741, 'text': "So clearly it's bad.", 'start': 512.52, 'duration': 1.221}, {'end': 515.501, 'text': 'So how do we improve on it, right?', 'start': 513.9, 'duration': 1.601}, {'end': 521.964, 'text': "Well, better than that, let's say let's, instead of saying you know, the next price is gonna be the previous day's price.", 'start': 515.841, 'duration': 6.123}], 'summary': "Naive strategy predicts next price as previous day's price, yielding poor results.", 'duration': 27.778, 'max_score': 494.186, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o494186.jpg'}, {'end': 563.779, 'src': 'embed', 'start': 535.531, 'weight': 2, 'content': [{'end': 539.314, 'text': 'So we can continue to improve this prediction strategy.', 'start': 535.531, 'duration': 3.783}, {'end': 542.376, 'text': "Instead of doing that, we can do what's called the moving average.", 'start': 539.494, 'duration': 2.882}, {'end': 548.54, 'text': "It's an average that we do over a certain amount of data points, and we continually add all of those up.", 'start': 542.956, 'duration': 5.584}, {'end': 551.163, 'text': 'and so that does a little better.', 'start': 549.441, 'duration': 1.722}, {'end': 554.848, 'text': "but we can, we can keep going, and in fact i've got a video that details this.", 'start': 551.163, 'duration': 3.685}, {'end': 556.47, 'text': "it's called time series prediction.", 'start': 554.848, 'duration': 1.622}, {'end': 559.093, 'text': 'search it on youtube, pause this, go, watch that, come back,', 'start': 556.47, 'duration': 2.623}, {'end': 563.779, 'text': "and you'll be a lot more knowledgeable about all the different ways we can predict on time series data.", 'start': 559.093, 'duration': 4.686}], 'summary': 'Improve prediction strategy with moving average for better results, detailed in a video on time series prediction.', 'duration': 28.248, 'max_score': 535.531, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o535531.jpg'}, {'end': 643.786, 'src': 'embed', 'start': 602.966, 'weight': 0, 'content': [{'end': 606.968, 'text': "So as you can see, it's not just time series prediction, there's more to it.", 'start': 602.966, 'duration': 4.002}, {'end': 610.069, 'text': "In fact, there are 10 steps and I'll talk about those in a second.", 'start': 607.288, 'duration': 2.781}, {'end': 614.01, 'text': "So as you see here, it's a two-step process in this graph example.", 'start': 610.589, 'duration': 3.421}, {'end': 621.593, 'text': 'We make a buy if the 50-day moving average is this, and then we make a sell if it goes below some threshold value.', 'start': 614.33, 'duration': 7.263}, {'end': 627.075, 'text': 'And these two simple instructions let us create our first algorithmic trading strategy.', 'start': 622.013, 'duration': 5.062}, {'end': 628.576, 'text': "But that's very simple.", 'start': 627.755, 'duration': 0.821}, {'end': 635.1, 'text': 'We can continually improve upon it with our knowledge of machine learning and of financial theory as well.', 'start': 628.616, 'duration': 6.484}, {'end': 643.786, 'text': "And the benefits of an algorithmic trading bot are that it's 100% mechanical, it's always running, there's no human involvement.", 'start': 636.141, 'duration': 7.645}], 'summary': 'Discussion on a two-step algorithmic trading process with 10 steps, leading to a 100% mechanical trading bot.', 'duration': 40.82, 'max_score': 602.966, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o602966.jpg'}], 'start': 373.983, 'title': 'Creating a trading bot and predicting time series data', 'summary': 'Covers creating a trading bot with deep reinforcement learning, implementing custom trading strategy, and predicting time series data in python using various strategies, emphasizing the importance of using predictions to make trading decisions and highlighting the prevalence and benefits of algorithmic trading in financial institutions.', 'chapters': [{'end': 436.354, 'start': 373.983, 'title': 'Creating a trading bot with deep reinforcement learning', 'summary': 'Covers the architecture of a trading bot, installation steps, and implementing a custom trading strategy using technical indicators and deep reinforcement learning, specifically actor critic, for making trades.', 'duration': 62.371, 'highlights': ['The chapter covers the architecture of a trading bot, including message passing between components and the installation process on a local machine.', 'Implementing a custom trading strategy using technical indicators and deep reinforcement learning, specifically actor critic, for making trades.', 'Explanation of a trading bot as an algorithm that makes trades based on repeatable instructions, with the goal of encoding these instructions to cover all edge cases.']}, {'end': 752.112, 'start': 436.374, 'title': 'Predicting time series data in python', 'summary': 'Discusses predicting time series data in python, starting with a naive strategy of predicting the price of tomorrow based on the price of yesterday, then progressing to more sophisticated prediction strategies such as simple average and moving average, emphasizing the importance of using these predictions to make trading decisions and highlighting the prevalence and benefits of algorithmic trading in financial institutions.', 'duration': 315.738, 'highlights': ['The chapter discusses the naive strategy of predicting time series data, starting with a simple Python code to import and preprocess dataset, then predicting the price of tomorrow based on the price of yesterday, resulting in poor prediction performance.', 'The chapter introduces more sophisticated prediction strategies, including simple average and moving average, with the moving average showing some improvement in prediction performance, highlighting the iterative nature of improving prediction strategies.', 'The chapter emphasizes the importance of using time series predictions to make trading decisions, including when to buy or sell based on trend predictions and setting stop loss limits, underlining the significance of leveraging predictions for financial decision-making.', 'The chapter highlights the prevalence of algorithmic trading in financial institutions, citing the use of trading bots in high-frequency trading and the widespread adoption of algorithmic trading strategies by banks, positioning knowledge of algorithmic trading as a competitive advantage.', 'The chapter underscores the benefits of algorithmic trading, including its mechanical nature, lack of emotional involvement, and its widespread use in high-frequency trading on Wall Street, providing insight into the advantages of employing algorithmic trading strategies.']}], 'duration': 378.129, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o373983.jpg', 'highlights': ['The chapter emphasizes the importance of using time series predictions to make trading decisions, including when to buy or sell based on trend predictions and setting stop loss limits, underlining the significance of leveraging predictions for financial decision-making.', 'The chapter underscores the benefits of algorithmic trading, including its mechanical nature, lack of emotional involvement, and its widespread use in high-frequency trading on Wall Street, providing insight into the advantages of employing algorithmic trading strategies.', 'The chapter introduces more sophisticated prediction strategies, including simple average and moving average, with the moving average showing some improvement in prediction performance, highlighting the iterative nature of improving prediction strategies.', 'The chapter discusses the naive strategy of predicting time series data, starting with a simple Python code to import and preprocess dataset, then predicting the price of tomorrow based on the price of yesterday, resulting in poor prediction performance.', 'The chapter covers the architecture of a trading bot, including message passing between components and the installation process on a local machine.', 'Implementing a custom trading strategy using technical indicators and deep reinforcement learning, specifically actor critic, for making trades.', 'Explanation of a trading bot as an algorithm that makes trades based on repeatable instructions, with the goal of encoding these instructions to cover all edge cases.', 'The chapter highlights the prevalence of algorithmic trading in financial institutions, citing the use of trading bots in high-frequency trading and the widespread adoption of algorithmic trading strategies by banks, positioning knowledge of algorithmic trading as a competitive advantage.']}, {'end': 1081.455, 'segs': [{'end': 860.152, 'src': 'embed', 'start': 829.554, 'weight': 1, 'content': [{'end': 832.015, 'text': "Well, let's just say Bitcoin in the cryptocurrency market.", 'start': 829.554, 'duration': 2.461}, {'end': 836.377, 'text': "Do we want to have multiple systems? Well, for now, let's just choose one to keep it simple.", 'start': 832.395, 'duration': 3.982}, {'end': 843.221, 'text': 'and then, once we decide all that, then we can build our strategy and then we perform back testing with it,', 'start': 836.977, 'duration': 6.244}, {'end': 852.467, 'text': 'which is analyzing its performance on historical price data, and this is a, you know, everybody that does this in the field back testing.', 'start': 843.221, 'duration': 9.246}, {'end': 860.152, 'text': "you got to do it if you're going to make any kind of useful algorithm, and once we've back tested it, it looks good for back testing.", 'start': 852.467, 'duration': 7.685}], 'summary': 'Choose one cryptocurrency system, develop strategy, perform backtesting.', 'duration': 30.598, 'max_score': 829.554, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o829554.jpg'}, {'end': 919.976, 'src': 'embed', 'start': 893.041, 'weight': 0, 'content': [{'end': 900.245, 'text': 'But in the machine learning framework, in the machine learning paradigm, I think, instead of choosing technical indicators,', 'start': 893.041, 'duration': 7.204}, {'end': 902.587, 'text': 'we can literally learn everything.', 'start': 900.245, 'duration': 2.342}, {'end': 905.108, 'text': 'We can have an algorithm learn what paradigm to use.', 'start': 903.027, 'duration': 2.081}, {'end': 909.191, 'text': 'We can have an algorithm learn what the technical indicators it should use should be.', 'start': 905.128, 'duration': 4.063}, {'end': 912.493, 'text': 'We can have an algorithm learn what the stop loss should be.', 'start': 909.611, 'duration': 2.882}, {'end': 914.734, 'text': 'We can have an algorithm learn everything.', 'start': 912.673, 'duration': 2.061}, {'end': 919.976, 'text': "And so it's a way of thinking about things where you have these weight values that are constantly being updated right?", 'start': 915.174, 'duration': 4.802}], 'summary': 'In machine learning, algorithms can learn technical indicators, stop loss, and more.', 'duration': 26.935, 'max_score': 893.041, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o893041.jpg'}, {'end': 1017.866, 'src': 'embed', 'start': 974.551, 'weight': 2, 'content': [{'end': 978.254, 'text': 'And let me just show you one more thing on the spot, like about training.', 'start': 974.551, 'duration': 3.703}, {'end': 980.596, 'text': 'This is a neural net in the browser.', 'start': 979.255, 'duration': 1.341}, {'end': 987.762, 'text': 'And so what I mean by this learning process is basically this.', 'start': 982.398, 'duration': 5.364}, {'end': 992.245, 'text': "As you can see, it's moving, data's moving here, and that's the neural net.", 'start': 988.763, 'duration': 3.482}, {'end': 993.905, 'text': 'Definitely check out this playground, by the way.', 'start': 992.305, 'duration': 1.6}, {'end': 997.967, 'text': 'It helps you learn about how they work a lot easier with the visual example.', 'start': 994.766, 'duration': 3.201}, {'end': 1000.188, 'text': "So let's go through the architecture of our code.", 'start': 998.067, 'duration': 2.121}, {'end': 1003.79, 'text': "Let's first of all just download this code and then we'll start running it all right?", 'start': 1000.508, 'duration': 3.282}, {'end': 1011.314, 'text': 'So the first step for us is to go to this repository called Gecko that I found on GitHub and it seems to be maintained pretty well.', 'start': 1003.97, 'duration': 7.344}, {'end': 1013.895, 'text': 'You know, it was last updated three days ago, which is good.', 'start': 1011.434, 'duration': 2.461}, {'end': 1017.866, 'text': "And we'll download this code repository.", 'start': 1015.202, 'duration': 2.664}], 'summary': 'Demonstrates neural net training in browser with visual example, using code from gecko repository.', 'duration': 43.315, 'max_score': 974.551, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o974551.jpg'}], 'start': 752.112, 'title': 'Building a trading algorithm', 'summary': 'Explores the process of building a trading algorithm for cryptocurrency, including market selection, strategy development, back testing, and the potential use of machine learning for algorithm creation.', 'chapters': [{'end': 1081.455, 'start': 752.112, 'title': 'Building a trading algorithm', 'summary': 'Discusses the process of building a trading algorithm for the cryptocurrency market, covering market selection, strategy development, back testing, and the potential of using machine learning for algorithm creation.', 'duration': 329.343, 'highlights': ['The process of building a trading algorithm for the cryptocurrency market is outlined, including market selection, strategy development, back testing, and real-time execution. Provides an overview of the key steps involved in building a trading algorithm, emphasizing the focus on the cryptocurrency market and the importance of back testing.', 'The potential of using machine learning to develop algorithms that can learn various factors, including the choice of trading strategy, technical indicators, and stop-loss levels. Introduces the concept of using machine learning to enable algorithms to learn multiple aspects of trading, illustrating the potential for a more adaptive and dynamic approach.', 'The speaker discusses using a neural net to demonstrate the learning process and emphasizes the visual example for easier understanding. Highlights the utilization of a neural net to showcase the learning process, with a focus on providing a visual example to aid comprehension.', 'The process of downloading and setting up a code repository for developing the trading algorithm using Gecko from GitHub is described. Details the steps involved in downloading and setting up a code repository from GitHub for initiating the development of the trading algorithm.']}], 'duration': 329.343, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o752112.jpg', 'highlights': ['The potential of using machine learning to develop algorithms for cryptocurrency trading, including strategy, indicators, and stop-loss levels.', 'The process of building a trading algorithm for the cryptocurrency market, emphasizing market selection, strategy development, back testing, and real-time execution.', 'The process of downloading and setting up a code repository for developing the trading algorithm using Gecko from GitHub is described.', 'The utilization of a neural net to showcase the learning process, with a focus on providing a visual example for easier understanding.']}, {'end': 1789.859, 'segs': [{'end': 1135.507, 'src': 'embed', 'start': 1107.18, 'weight': 0, 'content': [{'end': 1112.941, 'text': "So what are the important parts of this code base? It turns out that for the front end, it's using what's called Vue.js.", 'start': 1107.18, 'duration': 5.761}, {'end': 1122.943, 'text': "Like I said, it's this idea of reusable components similar to React that makes it easier to build scalable user interfaces in JavaScript.", 'start': 1113.661, 'duration': 9.282}, {'end': 1127.044, 'text': "And for state management, it's using a technology called Redux.", 'start': 1124.123, 'duration': 2.921}, {'end': 1135.507, 'text': 'And so you can think of all of these circles in this graph as the path that a user takes through the different routes of a web app you know,', 'start': 1127.324, 'duration': 8.183}], 'summary': 'Code base uses vue.js for front end and redux for state management.', 'duration': 28.327, 'max_score': 1107.18, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1107180.jpg'}, {'end': 1209.21, 'src': 'embed', 'start': 1166.748, 'weight': 1, 'content': [{'end': 1174.113, 'text': 'we have a lot of config to do, reusable config files for whatever market we want to use, for whatever trading strategy we want to use.', 'start': 1166.748, 'duration': 7.365}, {'end': 1182.798, 'text': 'What are the parameters? What are the variables we want to use? And so Tom is the format that the Gecko library is using for that.', 'start': 1174.413, 'duration': 8.385}, {'end': 1186.981, 'text': "It's called Tom's, what's it called? Tom's Obvious Minimal Language.", 'start': 1183.599, 'duration': 3.382}, {'end': 1187.902, 'text': "It's pretty simple.", 'start': 1187.321, 'duration': 0.581}, {'end': 1188.862, 'text': "It's pretty human readable.", 'start': 1187.942, 'duration': 0.92}, {'end': 1190.684, 'text': "There's no learning curve, really.", 'start': 1189.503, 'duration': 1.181}, {'end': 1193.486, 'text': "It's like JSON or any of the other ones.", 'start': 1191.044, 'duration': 2.442}, {'end': 1196.267, 'text': "Okay, so that's the basic idea there.", 'start': 1194.446, 'duration': 1.821}, {'end': 1198.907, 'text': 'There are two basic concepts in this code base.', 'start': 1196.627, 'duration': 2.28}, {'end': 1201.108, 'text': "There's the market, and then there's the Gecko stream.", 'start': 1198.947, 'duration': 2.161}, {'end': 1209.21, 'text': 'The market basically pulls data from whatever market we want and then the Gecko stream will execute whatever trade we want based on that market,', 'start': 1201.248, 'duration': 7.962}], 'summary': "Developing reusable config files using tom's obvious minimal language for trading strategies. two main concepts: market and gecko stream.", 'duration': 42.462, 'max_score': 1166.748, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1166748.jpg'}, {'end': 1319.274, 'src': 'embed', 'start': 1295.963, 'weight': 3, 'content': [{'end': 1306.948, 'text': "And it's basically a way it's a formatted pretty way of denoting what the price is at a certain interval and what the change was, what the delta was,", 'start': 1295.963, 'duration': 10.985}, {'end': 1312.551, 'text': 'and we can input each candle into our algorithm and use that as a data point for trading.', 'start': 1306.948, 'duration': 5.603}, {'end': 1319.274, 'text': "So this is what's inside of a candle, the amount, the start, end, first, last, timestamp, et cetera.", 'start': 1313.191, 'duration': 6.083}], 'summary': 'Candlestick charting displays price intervals and changes for trading algorithm input.', 'duration': 23.311, 'max_score': 1295.963, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1295963.jpg'}, {'end': 1503.055, 'src': 'embed', 'start': 1477.265, 'weight': 5, 'content': [{'end': 1483.066, 'text': 'There are actually quite a lot of technical indicators that we can use as signals to our trading algorithm.', 'start': 1477.265, 'duration': 5.801}, {'end': 1485.327, 'text': "And there's a lot of theory behind this.", 'start': 1483.607, 'duration': 1.72}, {'end': 1489.128, 'text': 'I mean, these are terms that are not used in machine learning.', 'start': 1485.387, 'duration': 3.741}, {'end': 1496.91, 'text': "Force index, volume price trend, average true range, Keltner channel, I've never heard of any of these before.", 'start': 1489.568, 'duration': 7.342}, {'end': 1501.374, 'text': 'Vortex index, mass index, money flow index.', 'start': 1497.411, 'duration': 3.963}, {'end': 1503.055, 'text': 'There are quite a lot.', 'start': 1502.014, 'duration': 1.041}], 'summary': 'Various technical indicators, including force index and volume price trend, can be used as signals for trading algorithms.', 'duration': 25.79, 'max_score': 1477.265, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1477265.jpg'}, {'end': 1632.257, 'src': 'embed', 'start': 1607.092, 'weight': 4, 'content': [{'end': 1613.138, 'text': "In reinforcement learning, what we first do is we frame our problem as what's called a Markov decision process.", 'start': 1607.092, 'duration': 6.046}, {'end': 1619.223, 'text': "This is a frame, it's a frame of thoughts to help us reason about what the algorithm we want to use should be like.", 'start': 1613.158, 'duration': 6.065}, {'end': 1630.215, 'text': 'So the Markov decision process involves states, actions and rewards in an environment, in any kind of simulated environment, where time is the label,', 'start': 1619.684, 'duration': 10.531}, {'end': 1632.257, 'text': 'where time is this continuous process?', 'start': 1630.215, 'duration': 2.042}], 'summary': 'Reinforcement learning uses markov decision process with states, actions, and rewards in a simulated environment.', 'duration': 25.165, 'max_score': 1607.092, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1607092.jpg'}], 'start': 1082.998, 'title': 'Gecko code base and candle data in trading', 'summary': "Provides an overview of the gecko code base, including its use of vue.js, redux, and tom's obvious minimal language, and explains the significance of candle data and deep reinforcement learning in trading algorithms, with insights into their applications in financial markets.", 'chapters': [{'end': 1277.427, 'start': 1082.998, 'title': 'Gecko code base overview', 'summary': "Provides an overview of the gecko code base, outlining its use of vue.js for the front end, redux for state management, and tom's obvious minimal language for serialization and configuration, along with its structure involving markets, gecko stream, adapters, plugins, and strategies.", 'duration': 194.429, 'highlights': ['The code base uses Vue.js for front end and Redux for state management. Vue.js and Redux are utilized for scalable user interfaces in JavaScript.', "The code base employs Tom's Obvious Minimal Language for serialization and configuration. Tom's Obvious Minimal Language is used for human-readable and reusable configuration files.", 'The structure involves markets, Gecko stream, adapters, plugins, and strategies. The code base includes components for pulling data from markets, executing trades, and utilizing various plugins and strategies.']}, {'end': 1789.859, 'start': 1278.939, 'title': 'Understanding candle data and deep reinforcement learning', 'summary': 'Explains the significance of candle data in trading algorithms and delves into the concept of deep reinforcement learning, outlining the key components and applications in financial markets.', 'duration': 510.92, 'highlights': ['The chapter explains the significance of candle data in trading algorithms and delves into the concept of deep reinforcement learning. The chapter elaborates on the use of candles as formatted representations of price changes and the utilization of candle data points for trading algorithms. It also introduces the concept of deep reinforcement learning as a cutting-edge AI technique applied in financial markets.', 'Reinforcement learning is framed as a Markov decision process, involving states, actions, rewards, and environments, with the application of neural networks to estimate policies over time. The chapter details the Markov decision process in reinforcement learning, encompassing states, actions, rewards, and environments. It also emphasizes the use of neural networks to estimate policies, citing examples such as the DeepQ network used by DeepMind for Atari games and AlphaGo by OpenAI.', 'The chapter introduces technical indicators as signals for trading algorithms, emphasizing the selection and testing of indicators for optimizing trading models. The chapter introduces various technical indicators such as force index, volume price trend, average true range, and vortex index, highlighting the process of selecting and testing indicators to optimize trading algorithms.']}], 'duration': 706.861, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1082998.jpg', 'highlights': ['The code base uses Vue.js for front end and Redux for state management. Vue.js and Redux are utilized for scalable user interfaces in JavaScript.', "The code base employs Tom's Obvious Minimal Language for serialization and configuration. Tom's Obvious Minimal Language is used for human-readable and reusable configuration files.", 'The structure involves markets, Gecko stream, adapters, plugins, and strategies. The code base includes components for pulling data from markets, executing trades, and utilizing various plugins and strategies.', 'The chapter explains the significance of candle data in trading algorithms and delves into the concept of deep reinforcement learning. The chapter elaborates on the use of candles as formatted representations of price changes and the utilization of candle data points for trading algorithms. It also introduces the concept of deep reinforcement learning as a cutting-edge AI technique applied in financial markets.', 'Reinforcement learning is framed as a Markov decision process, involving states, actions, rewards, and environments, with the application of neural networks to estimate policies over time. The chapter details the Markov decision process in reinforcement learning, encompassing states, actions, rewards, and environments. It also emphasizes the use of neural networks to estimate policies, citing examples such as the DeepQ network used by DeepMind for Atari games and AlphaGo by OpenAI.', 'The chapter introduces technical indicators as signals for trading algorithms, emphasizing the selection and testing of indicators for optimizing trading models. The chapter introduces various technical indicators such as force index, volume price trend, average true range, and vortex index, highlighting the process of selecting and testing indicators to optimize trading algorithms.']}, {'end': 2096.708, 'segs': [{'end': 1819.072, 'src': 'embed', 'start': 1789.859, 'weight': 1, 'content': [{'end': 1791.56, 'text': 'redo the process over and over and over again.', 'start': 1789.859, 'duration': 1.701}, {'end': 1793.56, 'text': "So, that's reinforcement learning.", 'start': 1791.84, 'duration': 1.72}, {'end': 1798.622, 'text': 'We can then split reinforcement learning into two types, model-based and model-free.', 'start': 1793.58, 'duration': 5.042}, {'end': 1807.145, 'text': "Now, model-based, it's good for, you know, We have a set model of the world, and you know, we're going to predict a lot.", 'start': 1799.222, 'duration': 7.923}, {'end': 1813.209, 'text': "But I think the more interesting techniques are model free, there's no model of the world, it builds that for itself.", 'start': 1807.565, 'duration': 5.644}, {'end': 1819.072, 'text': 'And we can further subdivide these model free methods into two types, value based and policy based.', 'start': 1813.609, 'duration': 5.463}], 'summary': 'Reinforcement learning involves model-based and model-free techniques, including value-based and policy-based methods.', 'duration': 29.213, 'max_score': 1789.859, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1789859.jpg'}, {'end': 1873.387, 'src': 'embed', 'start': 1848.733, 'weight': 2, 'content': [{'end': 1854.436, 'text': 'The policy uses the value function to decide, you know, what move to take.', 'start': 1848.733, 'duration': 5.703}, {'end': 1860.319, 'text': 'So there are actually two functions involved here, both a policy function, which is high level, and then a value function, which is low level.', 'start': 1854.456, 'duration': 5.863}, {'end': 1864.382, 'text': "Assess all of the values of all the pieces, and that's the value function.", 'start': 1860.62, 'duration': 3.762}, {'end': 1865.502, 'text': 'So a value.', 'start': 1864.862, 'duration': 0.64}, {'end': 1871.485, 'text': 'and then the policy decides okay, based on that assessment, which is the best one? Which is the highest value? Okay,', 'start': 1865.502, 'duration': 5.983}, {'end': 1873.387, 'text': "this one? That's the one we're going to take.", 'start': 1871.485, 'duration': 1.902}], 'summary': 'The policy uses a value function to assess and decide the best move based on the highest value.', 'duration': 24.654, 'max_score': 1848.733, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1848733.jpg'}, {'end': 1908.311, 'src': 'embed', 'start': 1883.573, 'weight': 0, 'content': [{'end': 1890.918, 'text': 'But it turns out that both value-based methods and policy-based methods, there are some methods that only focus on the value function.', 'start': 1883.573, 'duration': 7.345}, {'end': 1893.46, 'text': 'There are some methods that only focus on the policy function.', 'start': 1891.158, 'duration': 2.302}, {'end': 1897.563, 'text': "The better version of this is to combine both, and that's actor-critic.", 'start': 1894.34, 'duration': 3.223}, {'end': 1899.124, 'text': "It's a hybrid approach to both.", 'start': 1897.663, 'duration': 1.461}, {'end': 1902.327, 'text': 'So how does this work? Well, we have two neural networks.', 'start': 1899.605, 'duration': 2.722}, {'end': 1904.929, 'text': 'We have a critic network, and then we have an actor network.', 'start': 1902.367, 'duration': 2.562}, {'end': 1908.311, 'text': 'What the actor network does is it controls how our agent behaves.', 'start': 1905.089, 'duration': 3.222}], 'summary': 'Actor-critic is a hybrid approach using both value-based and policy-based methods with two neural networks.', 'duration': 24.738, 'max_score': 1883.573, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1883573.jpg'}, {'end': 1982.603, 'src': 'embed', 'start': 1941.234, 'weight': 3, 'content': [{'end': 1944.877, 'text': 'Again, gradient descent, Suraj, if you want to see how that optimization process works.', 'start': 1941.234, 'duration': 3.643}, {'end': 1946.558, 'text': "So that's the basic idea there.", 'start': 1945.437, 'duration': 1.121}, {'end': 1954.724, 'text': 'Mastering this architecture is essential to understanding state of the art algorithms like proximal policy optimization,', 'start': 1946.878, 'duration': 7.846}, {'end': 1958.266, 'text': 'which OpenAI released very recently, and others.', 'start': 1954.724, 'duration': 3.542}, {'end': 1961.368, 'text': "But the idea is that the actor says, I'm going to rotate this piece.", 'start': 1958.666, 'duration': 2.702}, {'end': 1964.01, 'text': "And then the critic is like, that's really bad or that's really good.", 'start': 1961.448, 'duration': 2.562}, {'end': 1965.211, 'text': "That's the critic part.", 'start': 1964.39, 'duration': 0.821}, {'end': 1971.295, 'text': 'And then mathematically speaking, we can say here are the equations that represent both the actor and the critic.', 'start': 1965.691, 'duration': 5.604}, {'end': 1972.636, 'text': 'The first is the actor.', 'start': 1971.696, 'duration': 0.94}, {'end': 1973.577, 'text': 'The second is the critic.', 'start': 1972.796, 'duration': 0.781}, {'end': 1976.479, 'text': 'And these are two weight values.', 'start': 1974.237, 'duration': 2.242}, {'end': 1978.48, 'text': 'So state, action, and theta.', 'start': 1976.519, 'duration': 1.961}, {'end': 1980.301, 'text': 'Theta is the weight value for that network.', 'start': 1978.54, 'duration': 1.761}, {'end': 1982.603, 'text': 'State, action, W.', 'start': 1980.722, 'duration': 1.881}], 'summary': 'Understanding gradient descent and architecture is essential for state-of-the-art algorithms like proximal policy optimization, recently released by openai.', 'duration': 41.369, 'max_score': 1941.234, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1941234.jpg'}, {'end': 2030.771, 'src': 'embed', 'start': 2001.295, 'weight': 5, 'content': [{'end': 2003.457, 'text': "And they're all going to be learning at each time step.", 'start': 2001.295, 'duration': 2.162}, {'end': 2011.241, 'text': "And over time, you know, based on the actions that it takes, it's going to optimize itself and update its weight values.", 'start': 2004.097, 'duration': 7.144}, {'end': 2017.804, 'text': 'So in the context of our problem, how does this all play out? The environment is some time series data.', 'start': 2011.781, 'duration': 6.023}, {'end': 2020.506, 'text': 'That is our, you know, Bitcoin price data.', 'start': 2018.125, 'duration': 2.381}, {'end': 2027.85, 'text': "The observation that it's going to make is going to include the price data, any other technical indicators that we want to input.", 'start': 2020.926, 'duration': 6.924}, {'end': 2030.771, 'text': 'all those features okay to use machine learning terminology', 'start': 2027.85, 'duration': 2.921}], 'summary': 'Using time series data and machine learning to optimize weight values for bitcoin price predictions.', 'duration': 29.476, 'max_score': 2001.295, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o2001295.jpg'}, {'end': 2102.611, 'src': 'embed', 'start': 2076.369, 'weight': 6, 'content': [{'end': 2083.792, 'text': 'which basically modifies that OpenAI gym environment so that one of the environments is a real-time data feed from a market.', 'start': 2076.369, 'duration': 7.423}, {'end': 2085.893, 'text': "And then, so let's say it's Apple stock.", 'start': 2084.232, 'duration': 1.661}, {'end': 2089.833, 'text': "Then we'll create an agent called an actor critic, and then we'll do the same thing with that.", 'start': 2086.293, 'duration': 3.54}, {'end': 2092.415, 'text': 'And the result is gonna be several actions.', 'start': 2090.274, 'duration': 2.141}, {'end': 2093.985, 'text': 'like that.', 'start': 2093.666, 'duration': 0.319}, {'end': 2095.647, 'text': "alright. so that's the basic idea.", 'start': 2093.985, 'duration': 1.662}, {'end': 2096.708, 'text': 'but I found it more.', 'start': 2095.647, 'duration': 1.061}, {'end': 2102.611, 'text': 'I found it easier actually to build off of that gecko repository, because that UI is so amazing,', 'start': 2096.708, 'duration': 5.903}], 'summary': 'Modifying openai gym for real-time stock market data, using an actor critic agent, resulting in several actions.', 'duration': 26.242, 'max_score': 2076.369, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o2076369.jpg'}], 'start': 1789.859, 'title': 'Reinforcement learning techniques', 'summary': 'Discusses reinforcement learning, emphasizing model-free methods, value function, policy function, and the actor-critic approach, highlighting the importance of combining both value-based and policy-based methods for effective learning.', 'chapters': [{'end': 1921.862, 'start': 1789.859, 'title': 'Reinforcement learning techniques', 'summary': 'Discusses reinforcement learning, emphasizing model-free methods, value function, policy function, and the actor-critic approach, highlighting the importance of combining both value-based and policy-based methods for effective learning.', 'duration': 132.003, 'highlights': ['The actor-critic approach combines both value-based and policy-based methods for effective learning, utilizing two neural networks: the critic network and the actor network. The actor-critic approach is a hybrid approach to reinforcement learning, utilizing both value-based and policy-based methods, and it involves the use of two neural networks: the critic network and the actor network.', 'Reinforcement learning can be split into model-based and model-free techniques, with the model-free approach being more intriguing as it builds the model for itself. Reinforcement learning can be categorized into model-based and model-free techniques, with the latter being more intriguing as it autonomously builds the model of the world for decision-making.', 'The policy-based method involves a policy function that decides which action to take based on the state, while the value function is used to assess the value of each state given an action. The policy-based method utilizes a policy function to determine actions based on the state, and it also involves a value function to evaluate the value of each state given an action.']}, {'end': 2096.708, 'start': 1922.342, 'title': 'Actor-critic architecture for financial market prediction', 'summary': 'Explains the actor-critic architecture for financial market prediction, including the learning process, optimization, and key components like state, action, and weight values, and its application in a real-time data feed environment.', 'duration': 174.366, 'highlights': ['The actor-critic architecture is essential for understanding state-of-the-art algorithms like proximal policy optimization and others. The actor-critic architecture is crucial for comprehending modern algorithms, such as proximal policy optimization.', "The state, action, and weight values (theta and W) are fundamental components of the actor-critic architecture, representing the network's parameters that change over time as it learns. The state, action, and weight values (theta and W) are crucial components of the actor-critic architecture, representing the network's evolving parameters.", "The environment for the actor-critic model involves time series data, such as Bitcoin price data, with observations including price data and technical indicators, and actions of buy, sell, or hold, with rewards based on profit or loss. The actor-critic model's environment comprises time series data like Bitcoin prices, observations encompassing price data and technical indicators, and actions of buy, sell, or hold, with rewards based on profit or loss.", "The actor-critic architecture can be applied in a real-time data feed environment, such as modifying OpenAI's gym environment to use a real-time data feed from a financial market like Apple stock. The actor-critic architecture can be utilized in a real-time data feed environment, like modifying OpenAI's gym environment to use real-time data from a financial market like Apple stock."]}], 'duration': 306.849, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o1789859.jpg', 'highlights': ['The actor-critic approach combines both value-based and policy-based methods for effective learning, utilizing two neural networks: the critic network and the actor network.', 'Reinforcement learning can be split into model-based and model-free techniques, with the model-free approach being more intriguing as it builds the model for itself.', 'The policy-based method involves a policy function that decides which action to take based on the state, while the value function is used to assess the value of each state given an action.', 'The actor-critic architecture is essential for understanding state-of-the-art algorithms like proximal policy optimization and others.', "The state, action, and weight values (theta and W) are fundamental components of the actor-critic architecture, representing the network's parameters that change over time as it learns.", 'The environment for the actor-critic model involves time series data, such as Bitcoin price data, with observations including price data and technical indicators, and actions of buy, sell, or hold, with rewards based on profit or loss.', "The actor-critic architecture can be applied in a real-time data feed environment, such as modifying OpenAI's gym environment to use a real-time data feed from a financial market like Apple stock."]}, {'end': 2532.319, 'segs': [{'end': 2124.61, 'src': 'embed', 'start': 2096.708, 'weight': 0, 'content': [{'end': 2102.611, 'text': 'I found it easier actually to build off of that gecko repository, because that UI is so amazing,', 'start': 2096.708, 'duration': 5.903}, {'end': 2108.055, 'text': 'and then add in machine learning via tensorflow.js or convnet.js or what have you?', 'start': 2102.611, 'duration': 5.444}, {'end': 2111.197, 'text': "so let's get started with building our custom trading strategy.", 'start': 2108.055, 'duration': 3.142}, {'end': 2117.702, 'text': "so what we're first going to do is we're going to look through the config sample config.js file.", 'start': 2111.197, 'duration': 6.505}, {'end': 2124.61, 'text': "This is the sample.config.js file, and it's got a bunch of variables that are just sample variables.", 'start': 2118.283, 'duration': 6.327}], 'summary': 'Building a custom trading strategy using gecko repository and integrating machine learning with tensorflow.js or convnet.js.', 'duration': 27.902, 'max_score': 2096.708, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o2096708.jpg'}, {'end': 2188.186, 'src': 'embed', 'start': 2164.089, 'weight': 1, 'content': [{'end': 2171.776, 'text': "So we'll start off by saying, okay, so this is called deep RL dot JS, deep RL dot JS.", 'start': 2164.089, 'duration': 7.687}, {'end': 2175.895, 'text': "Okay, now we'll start off by saying, here's my strategy.", 'start': 2173.032, 'duration': 2.863}, {'end': 2177.716, 'text': 'Well, this is going to create our own strategy.', 'start': 2175.915, 'duration': 1.801}, {'end': 2183.441, 'text': "And then we'll also just do a console.log for hello world, just to make sure that we're doing everything properly here.", 'start': 2178.457, 'duration': 4.984}, {'end': 2188.186, 'text': "And once we have that, we'll go ahead and prepare our strategy with the init function.", 'start': 2184.102, 'duration': 4.084}], 'summary': 'Introduction to deep rl dot js and strategy preparation', 'duration': 24.097, 'max_score': 2164.089, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o2164089.jpg'}, {'end': 2261.836, 'src': 'heatmap', 'start': 2228.403, 'weight': 0.753, 'content': [{'end': 2229.765, 'text': 'So long is actually going to wait.', 'start': 2228.403, 'duration': 1.362}, {'end': 2231.747, 'text': "It's going to say, OK, I don't want to sell right now.", 'start': 2229.965, 'duration': 1.782}, {'end': 2239.335, 'text': "And then I'll have one more, which is the ending function, which is optimal for executing code after the completion of a back test.", 'start': 2232.468, 'duration': 6.867}, {'end': 2247.153, 'text': "or you know some, some strategy, and lastly, I'll export the strategy, and that's it.", 'start': 2239.335, 'duration': 7.818}, {'end': 2252.314, 'text': 'And then, once we have that in the config file, we can run this config.js function.', 'start': 2247.553, 'duration': 4.761}, {'end': 2261.836, 'text': "and what that does is it says we're going to run a back test without the UI directly from terminal using that config.js file and it's going to see what's in there.", 'start': 2252.314, 'duration': 9.522}], 'summary': 'Explanation of a process involving back testing, strategy export, and terminal execution.', 'duration': 33.433, 'max_score': 2228.403, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o2228403.jpg'}, {'end': 2394.152, 'src': 'embed', 'start': 2372.384, 'weight': 2, 'content': [{'end': 2381.066, 'text': "on trade, if you know, if our stop loss is this amount and we set that stop loss as a threshold, then we're going to either buy or sell.", 'start': 2372.384, 'duration': 8.682}, {'end': 2387.287, 'text': 'and those functions are listed inside of this right here, like if buy, do this, else go long.', 'start': 2381.066, 'duration': 6.221}, {'end': 2388.727, 'text': 'this thought advice says that.', 'start': 2387.287, 'duration': 1.44}, {'end': 2394.152, 'text': 'And so you can see this set of if-then statements that define this strategy of using this neural network.', 'start': 2389.267, 'duration': 4.885}], 'summary': 'A trading strategy using neural network involves setting stop loss and executing buy or sell orders based on specific thresholds and conditions.', 'duration': 21.768, 'max_score': 2372.384, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o2372384.jpg'}], 'start': 2096.708, 'title': 'Custom trading strategies', 'summary': 'Delves into building custom trading strategies with gecko ui, leveraging machine learning via tensorflow.js or convnet.js, creating custom trading strategies, implementing deep rl, and developing neural network-based trading strategies to improve accuracy and push the ideas of deep reinforcement learning.', 'chapters': [{'end': 2140.051, 'start': 2096.708, 'title': 'Building custom trading strategy with gecko ui', 'summary': "Discusses leveraging the gecko repository's ui to incorporate machine learning via tensorflow.js or convnet.js for building a custom trading strategy by editing the sample.config.js file to create a new config.js file.", 'duration': 43.343, 'highlights': ["Leveraging the gecko repository's UI to incorporate machine learning via tensorflow.js or convnet.js for building a custom trading strategy", 'Editing the sample.config.js file to create a new config.js file']}, {'end': 2328.792, 'start': 2140.071, 'title': 'Creating custom trading strategy', 'summary': "Discusses creating a custom trading strategy called deep rl, implementing functions for initializing, updating, checking, and ending the strategy, and running a backtest to evaluate the strategy's performance.", 'duration': 188.721, 'highlights': ['Creation of a custom trading strategy named deep RL and defining the necessary functions for initializing, updating, checking, and ending the strategy', 'Running a backtest using the configured strategy to evaluate its performance and analyzing the data using the SMMA indicator', 'Referencing an existing strategy implemented using convnet.js and examining its structure and indicators']}, {'end': 2532.319, 'start': 2329.033, 'title': 'Neural net trading strategy', 'summary': 'Discusses the implementation of a neural network-based trading strategy using statistical features and reinforcement learning, aiming to improve accuracy and implement actor critic functionality, ultimately aiming to push the ideas of deep reinforcement learning into human consciousness.', 'duration': 203.286, 'highlights': ['Neural network-based trading strategy involves computing statistical features as input to the model, utilizing stochastic gradient descent for optimization, and implementing reinforcement learning using if-then statements to define the strategy.', 'Implementation of deep reinforcement learning involves adding actor critic functionality and connecting the strategy to a trading platform for real trades.', 'Exploration of utilizing reinforce.js for actor critic functionality and the potential to incorporate additional technical indicators to enhance model accuracy and back test the strategy for performance evaluation.']}], 'duration': 435.611, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/myydDX-us4o/pics/myydDX-us4o2096708.jpg', 'highlights': ["Leveraging the gecko repository's UI to incorporate machine learning via tensorflow.js or convnet.js for building a custom trading strategy", 'Creation of a custom trading strategy named deep RL and defining the necessary functions for initializing, updating, checking, and ending the strategy', 'Neural network-based trading strategy involves computing statistical features as input to the model, utilizing stochastic gradient descent for optimization, and implementing reinforcement learning using if-then statements to define the strategy.']}], 'highlights': ['The backtesting of the automated trading bot resulted in a 12% profit over a three-month span, with the algorithm making buy and sell decisions based on historical Bitcoin prices.', 'Different strategies, such as PPO and RSI, were tested on historical data, with the RSI strategy proving to be the most successful, yielding a 6% profit.', 'Automated live trading involves setting up API keys and choosing the exchange for real-time trades, such as Binance or Bitfinex.', 'Building a deep reinforcement learning agent for financial market prediction using cutting edge AI technology.', 'The chapter emphasizes the importance of using time series predictions to make trading decisions, including when to buy or sell based on trend predictions and setting stop loss limits, underlining the significance of leveraging predictions for financial decision-making.', 'The potential of using machine learning to develop algorithms for cryptocurrency trading, including strategy, indicators, and stop-loss levels.', 'The code base uses Vue.js for front end and Redux for state management. Vue.js and Redux are utilized for scalable user interfaces in JavaScript.', 'The actor-critic approach combines both value-based and policy-based methods for effective learning, utilizing two neural networks: the critic network and the actor network.', "Leveraging the gecko repository's UI to incorporate machine learning via tensorflow.js or convnet.js for building a custom trading strategy"]}