title
Data Warehouse Tutorial For Beginners | Data Warehouse Concepts | Data Warehousing | Edureka

description
🔥 Data Warehousing & BI Training (𝐔𝐬𝐞 𝐂𝐨𝐝𝐞: 𝐘𝐎𝐔𝐓𝐔𝐁𝐄𝟐𝟎): https://www.edureka.co/search This Data Warehouse Tutorial For Beginners will give you an introduction to data warehousing and business intelligence. You will be able to understand basic data warehouse concepts with examples. The following topics have been covered in this tutorial: 1. What Is The Need For BI? 2. What Is Data Warehousing? 3. Key Terminologies Related To DWH Architecture: a. OLTP Vs OLAP b. ETL c. Data Mart d. Metadata 4. DWH Architecture 5. Demo: Creating A DWH - - - - - - - - - - - - - - Check our complete Data Warehousing & Business Intelligence playlist here: https://goo.gl/DZEuZt. #DataWarehousing #DataWarehouseTutorial #DataWarehouseTraining Subscribe to our channel to get video updates. Hit the subscribe button above. - - - - - - - - - - - - - - How it Works? 1. This is a 5 Week Instructor led Online Course, 25 hours of assignment and 10 hours of project work 2. We have a 24x7 One-on-One LIVE Technical Support to help you with any problems you might face or any clarifications you may require during the course. 3. At the end of the training you will have to undergo a 2-hour LIVE Practical Exam based on which we will provide you a Grade and a Verifiable Certificate! - - - - - - - - - - - - - - About the Course: Edureka's Data Warehousing and Business Intelligence Course, will introduce participants to create and work with leading ETL & BI tools like: 1. Talend 5.x to create, execute, monitor and schedule ETL processes. It will cover concepts around Data Replication, Migration and Integration Operations 2. Tableau 9.x for data visualization to see how easy and reliable data visualization can become for representation with dashboards 3. Data Modeling tool ERwin r9 to create a Data Warehouse or Data Mart - - - - - - - - - - - - - - Who should go for this course? The following professionals can go for this course: 1. Data warehousing enthusiasts 2. Analytics Managers 3. Data Modelers 4. ETL Developers and BI Developers - - - - - - - - - - - - - - Why learn Data Warehousing and Business Intelligence? All the successful companies have been investing large sums of money in business intelligence and data warehousing tools and technologies. Up-to-date, accurate and integrated information about their supply chain, products and customers are critical for their success. With the advent of Mobile, Social and Cloud platform, today's business intelligence tools have evolved and can be categorized into five areas, including databases, extraction transformation and load (ETL) tools, data quality tools, reporting tools and statistical analysis tools. This course will provide a strong foundation around Data Warehousing and Business Intelligence fundamentals and sophisticated tools like Talend, Tableau and ERwin. - - - - - - - - - - - - - - For more information, please write back to us at sales@edureka.co or call us at IND: 9606058406 / US: 18338555775 (toll-free). Facebook: https://www.facebook.com/edurekaIN/ Twitter: https://twitter.com/edurekain LinkedIn: https://www.linkedin.com/company/edureka - - - - - - - - - - - - - - Customer Review: Kanishk says, "Underwent Mastering in DW-BI Course. The training material and trainer are up to the mark to get yourself acquainted to the new technology. Very helpful support service from Edureka."

detail
{'title': 'Data Warehouse Tutorial For Beginners | Data Warehouse Concepts | Data Warehousing | Edureka', 'heatmap': [{'end': 771.288, 'start': 646.908, 'weight': 0.828}, {'end': 1779.453, 'start': 1659.339, 'weight': 0.876}, {'end': 2313.06, 'start': 2250.353, 'weight': 0.932}, {'end': 2790.552, 'start': 2725.928, 'weight': 0.779}, {'end': 3086.598, 'start': 3019.483, 'weight': 1}], 'summary': 'This tutorial provides a comprehensive understanding of data warehousing and business intelligence, covering key concepts like oltp and olap, etl strategy, advantages of data warehousing, and practical demonstrations using tools like talend and oracle sql, with examples from successful companies like microsoft, google, and amazon.', 'chapters': [{'end': 155.533, 'segs': [{'end': 68.737, 'src': 'embed', 'start': 32.694, 'weight': 0, 'content': [{'end': 38.355, 'text': 'So business intelligence is one of the most important aspects for any company to grow well and to do good right?', 'start': 32.694, 'duration': 5.661}, {'end': 42.195, 'text': 'And data warehousing is among the most important activities of business intelligence.', 'start': 38.595, 'duration': 3.6}, {'end': 46.116, 'text': "So that's why these two things are interlinked and that's the connection these two have.", 'start': 42.815, 'duration': 3.301}, {'end': 50.937, 'text': 'So you can think of data warehousing to be a kind of a subset of business intelligence.', 'start': 46.616, 'duration': 4.321}, {'end': 57.338, 'text': 'So I will talk about these two things and after that, I will talk about the key terminologies that are related to data warehousing architecture.', 'start': 51.837, 'duration': 5.501}, {'end': 63.329, 'text': 'right, and some of the key terminologies are those of OLTP and OLAP, the differences between the two.', 'start': 58.021, 'duration': 5.308}, {'end': 68.737, 'text': 'okay, the OLTP, somewhat very similar to the databases, and OLAP is what represents data warehousing.', 'start': 63.329, 'duration': 5.408}], 'summary': 'Business intelligence and data warehousing are interlinked, with oltp and olap being key terminologies.', 'duration': 36.043, 'max_score': 32.694, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM832694.jpg'}, {'end': 111.5, 'src': 'embed', 'start': 84.829, 'weight': 2, 'content': [{'end': 89.11, 'text': 'ETL is a strategy to convert your data from your database onto your data warehouse, right?', 'start': 84.829, 'duration': 4.281}, {'end': 93.231, 'text': "So moving the data from one place to another, that's all done by ETL.", 'start': 89.21, 'duration': 4.021}, {'end': 94.812, 'text': "So we'll talk about ETL in detail.", 'start': 93.231, 'duration': 1.581}, {'end': 97.593, 'text': "All right, and after that I'll talk about what a data mart is.", 'start': 95.272, 'duration': 2.321}, {'end': 99.552, 'text': "So and then what's at meta data.", 'start': 97.633, 'duration': 1.919}, {'end': 103.915, 'text': 'Now these two things are two topics which I can only explain once I have given you an introduction to the other topics.', 'start': 99.772, 'duration': 4.143}, {'end': 104.735, 'text': 'All right.', 'start': 104.175, 'duration': 0.56}, {'end': 107.277, 'text': 'so any of your any doubts that you have during the session,', 'start': 104.735, 'duration': 2.542}, {'end': 111.5, 'text': "you can ask me at that time and I'll clear them right away and once I'm done teaching about.", 'start': 107.277, 'duration': 4.223}], 'summary': "Etl is a strategy to move data from database to data warehouse. it's essential for data management.", 'duration': 26.671, 'max_score': 84.829, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM884829.jpg'}, {'end': 123.168, 'src': 'embed', 'start': 114.642, 'weight': 4, 'content': [{'end': 123.168, 'text': 'I will show you the complete architecture and the complete the lifecycle of data and what kind of insights your company can get and what kind of advantages you can get out of data warehousing right?', 'start': 114.642, 'duration': 8.526}], 'summary': 'Showcasing complete data architecture and insights for company advantages.', 'duration': 8.526, 'max_score': 114.642, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8114642.jpg'}], 'start': 0.069, 'title': 'Data warehousing & business intelligence', 'summary': 'Delves into the significance of business intelligence and data warehousing, covering key concepts like oltp and olap, etl strategy, data mart, metadata, and the comprehensive architecture and lifecycle of data in the context of data warehousing, culminating with a demonstration of creating a data warehouse.', 'chapters': [{'end': 155.533, 'start': 0.069, 'title': 'Data warehousing & business intelligence', 'summary': 'Discusses the importance of business intelligence and data warehousing, key terminologies like oltp and olap, etl strategy, data mart, metadata, and the complete architecture and lifecycle of data in the context of data warehousing, ending with a demonstration of creating a data warehouse.', 'duration': 155.464, 'highlights': ['The importance of business intelligence and data warehousing is discussed, emphasizing their interlinked nature and the role of data warehousing as a subset of business intelligence. Business intelligence is crucial for company growth, with data warehousing being a vital activity within it, demonstrating the interlinked nature of the two.', 'Key terminologies like OLTP, OLAP, and the differences between databases and data warehouses are explained. Explanation of OLTP and OLAP, highlighting the differences between databases and data warehouses, providing clarity on why data warehousing suits business intelligence more than a database.', 'The ETL strategy for converting data from a database to a data warehouse is detailed. Detailed explanation of the ETL strategy, emphasizing its role in converting data from a database to a data warehouse.', 'The concept of a data mart and metadata is introduced, with a promise to explain in detail after an introduction to other topics. Introduction of the concepts of data mart and metadata, with the intention to provide detailed explanations after an introduction to other topics.', 'The complete architecture and lifecycle of data in the context of data warehousing will be covered, including the insights and advantages companies can derive from data warehousing. Coverage of the complete architecture and lifecycle of data in data warehousing, highlighting the insights and advantages for companies.']}], 'duration': 155.464, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM869.jpg', 'highlights': ['Business intelligence and data warehousing are interlinked, with data warehousing being a vital subset of business intelligence.', 'Explanation of OLTP and OLAP, highlighting the differences between databases and data warehouses.', 'Detailed explanation of the ETL strategy for converting data from a database to a data warehouse.', 'Introduction of the concepts of data mart and metadata, with detailed explanations to follow.', 'Coverage of the complete architecture and lifecycle of data in data warehousing, highlighting the insights and advantages for companies.']}, {'end': 1129.438, 'segs': [{'end': 198.069, 'src': 'embed', 'start': 171.593, 'weight': 1, 'content': [{'end': 177.681, 'text': "So we'll understand why business intelligence and data warehousing are among the fundamental and the foundation for any company's success.", 'start': 171.593, 'duration': 6.088}, {'end': 185.412, 'text': 'So why do we have to go for business intelligence? Business intelligence is the activity which contributes to the growth of any company.', 'start': 178.662, 'duration': 6.75}, {'end': 189.861, 'text': 'And there are also so many MNCs which have been established over the past few decades.', 'start': 186.198, 'duration': 3.663}, {'end': 194.406, 'text': "Now, how did that happen? They just didn't happen by luck, right? So they were all small ideas.", 'start': 190.242, 'duration': 4.164}, {'end': 198.069, 'text': 'They were small companies that started with a small idea and then they grew bigger.', 'start': 194.786, 'duration': 3.283}], 'summary': 'Business intelligence and data warehousing are fundamental for company success, as evidenced by the growth of mncs over the past few decades.', 'duration': 26.476, 'max_score': 171.593, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8171593.jpg'}, {'end': 258.93, 'src': 'embed', 'start': 234.857, 'weight': 0, 'content': [{'end': 241.939, 'text': 'So any company that has done well over the past few decades, be it Microsoft or Google, or Facebook or Amazon Salesforce,', 'start': 234.857, 'duration': 7.082}, {'end': 246.04, 'text': "all of these companies that have all grown from small ideas and they've become something big right?", 'start': 241.939, 'duration': 4.101}, {'end': 251.882, 'text': "And any startup that's also trying to do great nowadays, even they have to adopt the same strategy and the same plan.", 'start': 246.28, 'duration': 5.602}, {'end': 255.105, 'text': 'this is a very common thing and this is something that everyone knows.', 'start': 252.382, 'duration': 2.723}, {'end': 258.93, 'text': "okay, but this is not what I've, you know, come to teach all in this session.", 'start': 255.105, 'duration': 3.825}], 'summary': 'Successful companies like microsoft, google, facebook, and amazon have grown from small ideas, and startups today must adopt similar strategies.', 'duration': 24.073, 'max_score': 234.857, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8234857.jpg'}, {'end': 295.289, 'src': 'embed', 'start': 265.657, 'weight': 3, 'content': [{'end': 270.703, 'text': 'right so, before I talk about data warehousing, let me go into details of a business intelligence.', 'start': 265.657, 'duration': 5.046}, {'end': 272.405, 'text': 'so what exactly is business intelligence?', 'start': 270.703, 'duration': 1.702}, {'end': 278.705, 'text': 'bi is the act of transforming raw or operational data into useful information for business analysis.', 'start': 273.18, 'duration': 5.525}, {'end': 280.908, 'text': 'right so bi here stands for business intelligence.', 'start': 278.705, 'duration': 2.203}, {'end': 282.269, 'text': "that's the short form.", 'start': 280.908, 'duration': 1.361}, {'end': 285.272, 'text': 'and yeah, it is the act of transforming any raw or operational data.', 'start': 282.269, 'duration': 3.003}, {'end': 290.477, 'text': "so when we say raw or operational data, it's basically the data that you've collected, the data that you have about your business.", 'start': 285.272, 'duration': 5.205}, {'end': 295.289, 'text': "So it can be even if your company's starting from scratch, then whatever data you've gathered.", 'start': 291.027, 'duration': 4.262}], 'summary': 'Business intelligence transforms raw data into useful information for analysis.', 'duration': 29.632, 'max_score': 265.657, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8265657.jpg'}, {'end': 571.674, 'src': 'embed', 'start': 542.302, 'weight': 4, 'content': [{'end': 544.985, 'text': "that's where data warehouse comes in and that's where data warehouse goes.", 'start': 542.302, 'duration': 2.683}, {'end': 552.557, 'text': 'Data warehouse, it integrates data from all these databases and then processes that data and brings it in a form It is very easy to do visualization.', 'start': 545.586, 'duration': 6.971}, {'end': 555.122, 'text': "Okay, that's what the second point says.", 'start': 552.998, 'duration': 2.124}, {'end': 559.129, 'text': 'the data first needs to be integrated and then processed for visualization takes place.', 'start': 555.122, 'duration': 4.007}, {'end': 562.187, 'text': 'and Now this is the problem that you have with the regular databases.', 'start': 559.129, 'duration': 3.058}, {'end': 564.509, 'text': 'The data from here, they cannot be directly used for visualization.', 'start': 562.207, 'duration': 2.302}, {'end': 571.674, 'text': 'And since data warehouse can do that, since it can integrate data from multiple data warehouses and since that data can be processed easily,', 'start': 565.049, 'duration': 6.625}], 'summary': 'Data warehouse integrates and processes data for easy visualization.', 'duration': 29.372, 'max_score': 542.302, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8542302.jpg'}, {'end': 633.961, 'src': 'embed', 'start': 611.382, 'weight': 5, 'content': [{'end': 620.85, 'text': "Now a data warehouse is a central location where consolidated data from multiple locations or databases that's what locations mean from multiple locations are stored.", 'start': 611.382, 'duration': 9.468}, {'end': 623.292, 'text': 'Now this means, this is exactly what I explained earlier.', 'start': 620.97, 'duration': 2.322}, {'end': 625.694, 'text': "So you have data that's coming in from multiple data sources.", 'start': 623.352, 'duration': 2.342}, {'end': 629.357, 'text': 'You have all the data, you consolidate all the data into one single place.', 'start': 625.974, 'duration': 3.383}, {'end': 633.961, 'text': "And the data warehouse is maintained separately from an organization's operational database.", 'start': 629.857, 'duration': 4.104}], 'summary': 'A data warehouse stores consolidated data from multiple sources separately from operational databases.', 'duration': 22.579, 'max_score': 611.382, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8611382.jpg'}, {'end': 771.288, 'src': 'heatmap', 'start': 646.908, 'weight': 0.828, 'content': [{'end': 653.331, 'text': 'and the reason a data warehouse is stored separately from the operational database is because the data should not get affected.', 'start': 646.908, 'duration': 6.423}, {'end': 657.893, 'text': 'so you will have your operational data on one end, okay, where all your legacy data will be stored,', 'start': 653.331, 'duration': 4.562}, {'end': 660.054, 'text': 'where all your even probably in your real-time data will be stored.', 'start': 657.893, 'duration': 2.161}, {'end': 664.415, 'text': 'so all your transactions, all your sales, all your marketing, all your operations data,', 'start': 660.054, 'duration': 4.361}, {'end': 668.557, 'text': 'all these will be stored in one place and in during the act of data warehousing.', 'start': 664.415, 'duration': 4.142}, {'end': 673.719, 'text': "where you're doing that, when you're making analysis, when you're using that data, you don't want that to get corrupted, right.", 'start': 668.557, 'duration': 5.162}, {'end': 674.699, 'text': "so it's more like a backup.", 'start': 673.719, 'duration': 0.98}, {'end': 677.741, 'text': 'so for backup to our purpose, your operational data is separated.', 'start': 674.699, 'duration': 3.042}, {'end': 681.482, 'text': 'so you have your operational data, you keep it in one area and then you create a new database.', 'start': 677.741, 'duration': 3.741}, {'end': 683.363, 'text': "okay, in fact it's called a data warehouse.", 'start': 681.482, 'duration': 1.881}, {'end': 691.666, 'text': 'okay, so you get all the data from multiple sources or maybe from a single source, get it into a data warehouse and from here you do your analytics.', 'start': 683.363, 'duration': 8.303}, {'end': 695.387, 'text': 'so the process of getting the operational data into your data warehouse.', 'start': 691.666, 'duration': 3.721}, {'end': 697.908, 'text': "that's called extraction, transform and loading.", 'start': 695.387, 'duration': 2.521}, {'end': 704.231, 'text': "okay, now, when you've done these three things, you form your data warehouse and from your data warehouse you use the OLAP strategy.", 'start': 697.908, 'duration': 6.323}, {'end': 707.454, 'text': 'Okay, so OLAP stands for online analytical processing.', 'start': 704.491, 'duration': 2.963}, {'end': 713.678, 'text': 'So you use this OLAP strategy or well, this analytics processing for the business users to do analysis.', 'start': 707.674, 'duration': 6.004}, {'end': 716.38, 'text': "So it's there in the name, right? It stands for analytical processing.", 'start': 713.918, 'duration': 2.462}, {'end': 717.381, 'text': 'So the business users?', 'start': 716.42, 'duration': 0.961}, {'end': 718.421, 'text': 'what are analysis they want to do?', 'start': 717.381, 'duration': 1.04}, {'end': 723.865, 'text': 'They do it because there is the option of OLAP and then, along with the analysis, they can also do visualization.', 'start': 718.582, 'duration': 5.283}, {'end': 727.268, 'text': 'For visualization, you have various other tools like Tableau and ClickView.', 'start': 724.286, 'duration': 2.982}, {'end': 731.334, 'text': 'right. there are some amazing tools so you can get this data.', 'start': 727.788, 'duration': 3.546}, {'end': 734.199, 'text': 'you can get it into the data warehouse and the data warehouse also.', 'start': 731.334, 'duration': 2.865}, {'end': 735.681, 'text': 'it can be stored in some kind of database.', 'start': 734.199, 'duration': 1.482}, {'end': 743.139, 'text': "it can store this data back into some kind of oracle or sql server, or maybe even an excel, and when you've stored there,", 'start': 735.681, 'duration': 7.458}, {'end': 750.723, 'text': 'then you can do your OLAP activities there and also you can import that data into your various visualization tools like Tableau or ClickView,', 'start': 743.139, 'duration': 7.584}, {'end': 752.444, 'text': 'and thus you can get insights.', 'start': 750.723, 'duration': 1.721}, {'end': 754.385, 'text': 'You can get insights into your data.', 'start': 752.544, 'duration': 1.841}, {'end': 759.989, 'text': 'you can deliver presentations during your board meetings, you can show your findings to your superiors or your managers.', 'start': 754.385, 'duration': 5.604}, {'end': 760.809, 'text': 'you can do all these things.', 'start': 759.989, 'duration': 0.82}, {'end': 768.627, 'text': "So that's what a data warehouse is, okay? And then the next point we have here is, end users access it whenever any information is needed.", 'start': 760.969, 'duration': 7.658}, {'end': 771.288, 'text': 'Yeah, so this is again the same thing, right?', 'start': 768.907, 'duration': 2.381}], 'summary': 'Data warehouse stores operational data separately for analytics, enabling olap and visualization for insights and presentations.', 'duration': 124.38, 'max_score': 646.908, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8646908.jpg'}, {'end': 914.991, 'src': 'embed', 'start': 890.546, 'weight': 6, 'content': [{'end': 898.047, 'text': "So what you find in your data warehouse is legacy data, right? It's historical data which you can use to perform analysis or all those find insights.", 'start': 890.546, 'duration': 7.501}, {'end': 905.209, 'text': 'The operational data, if you have new data coming in here, this has to be imported and this has to be moved to your data warehouse first.', 'start': 898.627, 'duration': 6.582}, {'end': 910.65, 'text': "And then once it's moved to your data warehouse, from here it can be used for analysis and all these things by your end users.", 'start': 905.309, 'duration': 5.341}, {'end': 914.991, 'text': "So that's what this diagram here means and that's what the last point also means.", 'start': 910.93, 'duration': 4.061}], 'summary': 'Data warehouse stores legacy data for analysis, new data needs to be imported.', 'duration': 24.445, 'max_score': 890.546, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8890546.jpg'}, {'end': 952.09, 'src': 'embed', 'start': 926.743, 'weight': 7, 'content': [{'end': 932.726, 'text': 'then if you guys have understood what a data warehouses, I can go to my next slide and talk about the next topic.', 'start': 926.743, 'duration': 5.983}, {'end': 935.287, 'text': 'so that was about data warehouse.', 'start': 932.726, 'duration': 2.561}, {'end': 942.73, 'text': "now let's look at the advantages of a data warehouse when we compare it to any database or just regular flat files and all these things.", 'start': 935.287, 'duration': 7.443}, {'end': 947.188, 'text': 'The first advantage is that strategic questions can be answered by starting trends.', 'start': 943.666, 'duration': 3.522}, {'end': 949.729, 'text': 'So this is the biggest benefit that you can get.', 'start': 947.828, 'duration': 1.901}, {'end': 952.09, 'text': 'Your data analysts and data scientists.', 'start': 950.269, 'duration': 1.821}], 'summary': 'Data warehouse provides advantage of answering strategic questions by identifying trends, benefitting data analysts and scientists.', 'duration': 25.347, 'max_score': 926.743, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8926743.jpg'}], 'start': 156.313, 'title': 'Role of business intelligence and data warehousing', 'summary': 'Discusses the fundamental role of business intelligence and data warehousing in the success and growth of companies, emphasizing planning, data gathering, analysis, strategy formulation, and business action, with examples of successful companies like microsoft, google, facebook, amazon, and salesforce. it also explains the role of data warehousing in business intelligence, emphasizing the process of transforming and integrating raw data into a data warehouse and highlighting the advantages of a data warehouse over a database.', 'chapters': [{'end': 251.882, 'start': 156.313, 'title': 'Importance of business intelligence', 'summary': 'Discusses the fundamental role of business intelligence and data warehousing in the success and growth of companies, highlighting the process of planning, data gathering, analysis, strategy formulation, and business action, with examples of successful companies like microsoft, google, facebook, amazon, and salesforce.', 'duration': 95.569, 'highlights': ['Successful companies like Microsoft, Google, Facebook, Amazon, and Salesforce have grown from small ideas, emphasizing the importance of effective planning and data-driven strategies.', "Business intelligence and data warehousing are fundamental for any company's success, contributing to growth and enabling companies to recoup their investments.", 'The process of business intelligence involves planning, data gathering, analysis, strategy formulation, and business action, leading to company growth and success.']}, {'end': 860.487, 'start': 252.382, 'title': 'Data warehousing and business intelligence', 'summary': 'Explains the role of data warehousing in business intelligence, emphasizing the process of transforming and integrating raw data from various sources into a data warehouse, enabling easy visualization, analysis, and access for end users.', 'duration': 608.105, 'highlights': ['Data warehousing is the act of transforming raw or operational data into useful information for business analysis, forming a crucial part of business intelligence. Business intelligence involves transforming raw or operational data into useful information for business analysis, and data warehousing is a vital component of this process.', 'Data warehouse technology extracts, transforms, cleans, and integrates data from operational systems into a structured format for visualization and analysis, facilitating decision-making for end users. Data warehouse technology plays a key role in extracting, transforming, and integrating data from operational systems into a structured format, enabling visualization and analysis for informed decision-making by end users.', "Data warehouse serves as a central location for consolidated data from multiple sources, separate from an organization's operational database, enabling end users to access historical data for analysis and decision-making. A data warehouse acts as a centralized repository for consolidated data from multiple sources, distinct from an organization's operational database, allowing end users to access historical data for analysis and decision-making purposes."]}, {'end': 1129.438, 'start': 861.147, 'title': 'Understanding data warehousing', 'summary': 'Explains the concept of data warehousing, emphasizing the process of importing operational data into the data warehouse, and highlights the advantages of a data warehouse over a database, including the ability to answer strategic questions by analyzing trends and integrating data from multiple sources.', 'duration': 268.291, 'highlights': ['The process of importing operational data into the data warehouse is explained, emphasizing that only legacy and historical data are present in the data warehouse, and it needs to be imported before being used for analysis. ', 'The advantages of a data warehouse over a database are highlighted, including the ability to answer strategic questions by analyzing trends and integrating data from multiple sources, which provides a more structured and related data for business users. ']}], 'duration': 973.125, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM8156313.jpg', 'highlights': ['Successful companies like Microsoft, Google, Facebook, Amazon, and Salesforce emphasize the importance of effective planning and data-driven strategies.', "Business intelligence and data warehousing are fundamental for any company's success, contributing to growth and enabling companies to recoup their investments.", 'The process of business intelligence involves planning, data gathering, analysis, strategy formulation, and business action, leading to company growth and success.', 'Data warehousing is the act of transforming raw or operational data into useful information for business analysis, forming a crucial part of business intelligence.', 'Data warehouse technology plays a key role in extracting, transforming, and integrating data from operational systems into a structured format, enabling visualization and analysis for informed decision-making by end users.', "A data warehouse acts as a centralized repository for consolidated data from multiple sources, distinct from an organization's operational database, allowing end users to access historical data for analysis and decision-making purposes.", 'The process of importing operational data into the data warehouse is explained, emphasizing that only legacy and historical data are present in the data warehouse, and it needs to be imported before being used for analysis.', 'The advantages of a data warehouse over a database are highlighted, including the ability to answer strategic questions by analyzing trends and integrating data from multiple sources, which provides a more structured and related data for business users.']}, {'end': 1503.444, 'segs': [{'end': 1196.81, 'src': 'embed', 'start': 1166.628, 'weight': 0, 'content': [{'end': 1168.27, 'text': 'Processed data is called information.', 'start': 1166.628, 'duration': 1.642}, {'end': 1172.715, 'text': 'So information is easier to understand, easier to relate to, and easier to use.', 'start': 1168.789, 'duration': 3.926}, {'end': 1175.139, 'text': "Now that's what data warehouse does.", 'start': 1173.096, 'duration': 2.043}, {'end': 1178.705, 'text': 'It takes you one step closer to information, right?', 'start': 1175.259, 'duration': 3.446}, {'end': 1180.806, 'text': "So that's the advantage.", 'start': 1179.386, 'duration': 1.42}, {'end': 1184.407, 'text': "and yeah, the other thing is data browsing is faster and it's more accurate.", 'start': 1180.806, 'duration': 3.601}, {'end': 1189.408, 'text': "Yes This is something that's completely true because in your database, you will have loads of data.", 'start': 1184.787, 'duration': 4.621}, {'end': 1192.709, 'text': "You will of course, you'll have historical data and real-time data.", 'start': 1189.428, 'duration': 3.281}, {'end': 1196.81, 'text': "But the thing is it's not going to be as fast as our data warehouse data warehouse.", 'start': 1192.749, 'duration': 4.061}], 'summary': 'Data warehouse makes information easier to understand, relate to, and use, while also providing faster and more accurate data browsing.', 'duration': 30.182, 'max_score': 1166.628, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81166628.jpg'}, {'end': 1242.244, 'src': 'embed', 'start': 1217.489, 'weight': 3, 'content': [{'end': 1225.251, 'text': "So whatever processing or analysis it's done based on the past data that is stored in the data warehouse and that makes this data more accurate.", 'start': 1217.489, 'duration': 7.762}, {'end': 1226.551, 'text': 'It makes it more stable.', 'start': 1225.631, 'duration': 0.92}, {'end': 1231.833, 'text': 'So stability is the key word here, and stability is not something that you can have all the time in database,', 'start': 1226.972, 'duration': 4.861}, {'end': 1233.893, 'text': 'but you will have it with the data warehouse.', 'start': 1231.833, 'duration': 2.06}, {'end': 1235.714, 'text': "So that's the second big advantage.", 'start': 1233.933, 'duration': 1.781}, {'end': 1242.244, 'text': 'And in fact, there are many more advantages, right? So data warehousing is something that you guys will understand when you start implementing.', 'start': 1236.343, 'duration': 5.901}], 'summary': 'Data warehousing enhances data accuracy and stability, providing numerous advantages for implementation.', 'duration': 24.755, 'max_score': 1217.489, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81217489.jpg'}, {'end': 1311.006, 'src': 'embed', 'start': 1284.475, 'weight': 4, 'content': [{'end': 1288.736, 'text': "right. so data warehouse is more of a concept and it's a strategy, and it's not an end product.", 'start': 1284.475, 'duration': 4.261}, {'end': 1290.817, 'text': "it's not a tool or something that you can use.", 'start': 1288.736, 'duration': 2.081}, {'end': 1297.479, 'text': 'you have multiple tools to implement data warehousing and the thing you got to notice data warehousing is not a product okay.', 'start': 1290.817, 'duration': 6.662}, {'end': 1302.881, 'text': "so it's a strategy that you adopt to make your data more readable and make your data in a better fashion.", 'start': 1297.479, 'duration': 5.402}, {'end': 1306.502, 'text': "So that's the biggest advantage with data warehouse.", 'start': 1303.88, 'duration': 2.622}, {'end': 1308.244, 'text': 'So look at this guy here.', 'start': 1306.723, 'duration': 1.521}, {'end': 1311.006, 'text': "He'll just run one query on the data warehouse.", 'start': 1308.544, 'duration': 2.462}], 'summary': 'Data warehouse is a strategy to make data more readable and accessible, not a product or tool.', 'duration': 26.531, 'max_score': 1284.475, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81284475.jpg'}, {'end': 1343.819, 'src': 'embed', 'start': 1317.571, 'weight': 5, 'content': [{'end': 1327.42, 'text': 'then all those multiple data from multiple operational systems will be integrated together and then that will be standardized and any inconsistencies there in that data will be removed.', 'start': 1317.571, 'duration': 9.849}, {'end': 1329.882, 'text': 'Now these are the three important things.', 'start': 1328.34, 'duration': 1.542}, {'end': 1334.027, 'text': 'the data is taken from the operational systems and that data.', 'start': 1330.422, 'duration': 3.605}, {'end': 1337.451, 'text': 'if there are multiple operational systems, those would be integrated, okay,', 'start': 1334.027, 'duration': 3.424}, {'end': 1343.819, 'text': 'and then the data will be standardized and any inconsistencies will be removed, and once all these three things are done,', 'start': 1337.451, 'duration': 6.368}], 'summary': "Multiple operational systems' data integrated, standardized, and inconsistencies removed.", 'duration': 26.248, 'max_score': 1317.571, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81317571.jpg'}, {'end': 1421.416, 'src': 'embed', 'start': 1398.277, 'weight': 8, 'content': [{'end': 1405.726, 'text': 'Okay, so when we say subject oriented, it means that the data will be categorized and stored by the business subject rather than by the application.', 'start': 1398.277, 'duration': 7.449}, {'end': 1408.449, 'text': 'Now, let me get back to this point after I finish these three.', 'start': 1406.026, 'duration': 2.423}, {'end': 1415.793, 'text': "Okay, now this is the most complicated point, okay? Now, talking about integration, right? He said that data's integrated.", 'start': 1408.549, 'duration': 7.244}, {'end': 1421.416, 'text': 'So the meaning here is data on a given subject is collected from disparate sources and stored in a single place.', 'start': 1415.874, 'duration': 5.542}], 'summary': 'Data is stored by business subject, integrated from disparate sources.', 'duration': 23.139, 'max_score': 1398.277, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81398277.jpg'}, {'end': 1465.998, 'src': 'embed', 'start': 1435.644, 'weight': 6, 'content': [{'end': 1438.869, 'text': 'It is stored as a series of snapshots, each representing a period of time.', 'start': 1435.644, 'duration': 3.225}, {'end': 1443.737, 'text': 'So when you do your analysis, you can do it based on a series of snapshots of time.', 'start': 1439.33, 'duration': 4.407}, {'end': 1450.867, 'text': 'okay?. You can see what was your company status on this month, that year or on this month this year.', 'start': 1443.737, 'duration': 7.13}, {'end': 1452.528, 'text': "What is the progress that's been made?", 'start': 1451.367, 'duration': 1.161}, {'end': 1458.613, 'text': "Or if it's not a progress, if it's the same, if your growth has been stagnated, then you can find out what are the metrics.", 'start': 1452.828, 'duration': 5.785}, {'end': 1460.154, 'text': 'what are the reasons why that has happened?', 'start': 1458.613, 'duration': 1.541}, {'end': 1465.998, 'text': 'You can find all these things and you can look at all those details from a time approach, right, from a time variant approach.', 'start': 1460.614, 'duration': 5.384}], 'summary': 'Data is stored as series of snapshots, allowing analysis of company status and progress over time.', 'duration': 30.354, 'max_score': 1435.644, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81435644.jpg'}, {'end': 1490.255, 'src': 'embed', 'start': 1466.098, 'weight': 7, 'content': [{'end': 1472.143, 'text': "So that's what a data warehouse, the advantage here is, okay? That's one of the properties and the advantages that you have.", 'start': 1466.098, 'duration': 6.045}, {'end': 1474.802, 'text': 'And then data is non-volatile.', 'start': 1473.12, 'duration': 1.682}, {'end': 1477.664, 'text': 'The data in a data warehouse is not updated or deleted.', 'start': 1475.222, 'duration': 2.442}, {'end': 1480.306, 'text': 'So this is what is the other property that I mentioned earlier.', 'start': 1477.764, 'duration': 2.542}, {'end': 1485.391, 'text': 'Once the data comes into a data warehouse, it cannot be deleted or neither can it be changed.', 'start': 1480.967, 'duration': 4.424}, {'end': 1490.255, 'text': 'In fact, it can be updated, but the process of having to update it is a little complicated.', 'start': 1485.611, 'duration': 4.644}], 'summary': 'A data warehouse is non-volatile, data cannot be updated or deleted once it enters, only complex updates are possible.', 'duration': 24.157, 'max_score': 1466.098, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81466098.jpg'}], 'start': 1130.615, 'title': 'Advantages of data warehousing', 'summary': 'Discusses the advantages of data warehousing, including its ability to provide readable information, faster data browsing, improved data accuracy, and integration of data from multiple sources, ultimately leading to easier understanding and analysis. it also emphasizes the stability, customizability, and four important properties of data warehousing.', 'chapters': [{'end': 1215.45, 'start': 1130.615, 'title': 'Advantages of data warehouse', 'summary': 'Discusses the advantages of data warehouse, highlighting its ability to provide readable information, faster data browsing, and more accurate data, ultimately leading to easier understanding, relating to, and using of information.', 'duration': 84.835, 'highlights': ['Data warehouse provides readable information, making it easier to understand, relate to, and use, ultimately answering strategic questions and studying trends.', 'Data browsing in a data warehouse is faster and more accurate compared to a regular database, due to the presence of links, tables, and relations between various data, enabling easy access and gathering of information.', 'Processed data in a data warehouse is easier to understand, relate to, and use, making it one step closer to information, leading to better decision-making and insights.']}, {'end': 1302.881, 'start': 1217.489, 'title': 'Advantages of data warehousing', 'summary': 'Discusses the stability and customizability of data warehousing, emphasizing that it is a strategy for improving data accuracy and readability, rather than a product that can be purchased.', 'duration': 85.392, 'highlights': ['The stability and accuracy of data warehouse processing make it a key factor in improving data quality and stability, providing an advantage over traditional databases.', "Data warehousing is a customizable strategy that depends entirely on the company's requirements, making it a concept rather than a purchasable product.", 'Data warehousing is not a standalone tool or product that can be purchased; rather, it is a concept and strategy for optimizing data readability and quality.']}, {'end': 1503.444, 'start': 1303.88, 'title': 'Advantages of data warehousing', 'summary': 'Explains the advantages and properties of a data warehouse, highlighting that it integrates data from multiple sources, standardizes, removes inconsistencies, and stores it in an easy format for quick and accurate analysis. it also outlines the four important properties: subject-oriented, integrated, time-variant, and non-volatile data collection.', 'duration': 199.564, 'highlights': ['Data integration: Data from multiple operational systems is integrated, standardized, and inconsistencies are removed, providing easy access and analysis. Data from multiple operational systems is integrated and standardized, ensuring easy access and analysis.', 'Time-variant data: It is stored as a series of snapshots, allowing analysis based on different time periods for understanding company status, progress, and stagnation. Data is stored as a series of snapshots, enabling analysis based on different time periods for understanding company status and progress.', 'Non-volatile data: Once in the data warehouse, data cannot be updated or deleted, ensuring data integrity and preventing corruption. Data in a data warehouse is non-volatile, preventing updates or deletions to maintain data integrity and prevent corruption.', "Subject-oriented data: Data is categorized and stored by business subject rather than by the application, facilitating management's decision-making process. Data is categorized and stored by business subject, facilitating the management's decision-making process."]}], 'duration': 372.829, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81130615.jpg', 'highlights': ['Data warehousing provides readable information, making it easier to understand, relate to, and use, ultimately answering strategic questions and studying trends.', 'Data browsing in a data warehouse is faster and more accurate compared to a regular database, due to the presence of links, tables, and relations between various data, enabling easy access and gathering of information.', 'Processed data in a data warehouse is easier to understand, relate to, and use, making it one step closer to information, leading to better decision-making and insights.', 'The stability and accuracy of data warehouse processing make it a key factor in improving data quality and stability, providing an advantage over traditional databases.', "Data warehousing is a customizable strategy that depends entirely on the company's requirements, making it a concept rather than a purchasable product.", 'Data integration: Data from multiple operational systems is integrated, standardized, and inconsistencies are removed, providing easy access and analysis.', 'Time-variant data: It is stored as a series of snapshots, allowing analysis based on different time periods for understanding company status, progress, and stagnation.', 'Non-volatile data: Once in the data warehouse, data cannot be updated or deleted, ensuring data integrity and preventing corruption.', "Subject-oriented data: Data is categorized and stored by business subject rather than by the application, facilitating management's decision-making process."]}, {'end': 2206.445, 'segs': [{'end': 1534.667, 'src': 'embed', 'start': 1503.444, 'weight': 0, 'content': [{'end': 1508.667, 'text': "and that's why doing analysis and all these things are, you know, a better option.", 'start': 1503.444, 'duration': 5.223}, {'end': 1512.389, 'text': 'now, getting back to the first point, we said that it is subject oriented.', 'start': 1508.667, 'duration': 3.722}, {'end': 1516.831, 'text': 'right, data is categorized and stored by business subject rather than by the application.', 'start': 1512.389, 'duration': 4.442}, {'end': 1523.795, 'text': 'now, what this means is the data here will be stored, or the data that you will you know that you retrieve from a data warehouse.', 'start': 1516.831, 'duration': 6.964}, {'end': 1526.356, 'text': 'right, you will get it in the form that you wanted to.', 'start': 1523.795, 'duration': 2.561}, {'end': 1534.667, 'text': "Now, if you want to give me an example of that, let's say that we are dealing with a retail company, and in my retail company I have a marketing team,", 'start': 1527.065, 'duration': 7.602}], 'summary': 'Data warehouse stores data by business subject, ensuring retrieval in desired form.', 'duration': 31.223, 'max_score': 1503.444, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81503444.jpg'}, {'end': 1779.453, 'src': 'heatmap', 'start': 1659.339, 'weight': 0.876, 'content': [{'end': 1664.101, 'text': 'Why, you know, how important business intelligence is and what kind of role this data warehouse plays.', 'start': 1659.339, 'duration': 4.762}, {'end': 1669.08, 'text': 'you can just think of how humongous a deal a data warehouses, correct?', 'start': 1664.517, 'duration': 4.563}, {'end': 1673.224, 'text': 'yeah, so a couple of people are satisfied with that.', 'start': 1669.08, 'duration': 4.144}, {'end': 1676.326, 'text': "so now that we've spoken about the properties, let's go to the next slide.", 'start': 1673.224, 'duration': 3.102}, {'end': 1679.909, 'text': 'and okay, now we have to talk about key terminologies.', 'start': 1676.326, 'duration': 3.583}, {'end': 1684.052, 'text': 'okay, so, right now we want us to data warehousing from a higher level.', 'start': 1679.909, 'duration': 4.143}, {'end': 1685.333, 'text': "okay, now, let's dig deep.", 'start': 1684.052, 'duration': 1.281}, {'end': 1687.674, 'text': "let's go to more basics here.", 'start': 1685.333, 'duration': 2.341}, {'end': 1692.398, 'text': "okay, let's understand the key terminologies are related and that are involved in a data warehousing.", 'start': 1687.674, 'duration': 4.724}, {'end': 1696.199, 'text': 'So, first of all, we have OLTP and OLAP.', 'start': 1693.257, 'duration': 2.942}, {'end': 1698.12, 'text': 'Now there are four things I will talk about.', 'start': 1696.479, 'duration': 1.641}, {'end': 1706.706, 'text': "The differences between OLTP and OLAP, then I'll talk about EDL, I'll talk about data mart, and then finally about metadata.", 'start': 1698.66, 'duration': 8.046}, {'end': 1710.208, 'text': 'So let me go to the first topic, that is OLTP versus OLAP.', 'start': 1707.446, 'duration': 2.762}, {'end': 1717.112, 'text': 'So in this part, which is OLTP, OLTP stands for online transaction processing.', 'start': 1711.709, 'duration': 5.403}, {'end': 1721.595, 'text': 'Now this is something that is a representation of that of a database.', 'start': 1717.652, 'duration': 3.943}, {'end': 1727.454, 'text': "if you're running any kind of queries on your database, then that's called online transaction processing, okay?", 'start': 1722.068, 'duration': 5.386}, {'end': 1731.057, 'text': 'And then OLAP stands for online analytical processing.', 'start': 1727.914, 'duration': 3.143}, {'end': 1733.38, 'text': 'And this is the property of a data warehouse.', 'start': 1731.298, 'duration': 2.082}, {'end': 1740.107, 'text': "So any kind of query or any kind of analysis that you run on your data warehouse, that's called as an OLAP activity, correct?", 'start': 1733.88, 'duration': 6.227}, {'end': 1742.349, 'text': "So let's go to the differences between the two.", 'start': 1740.827, 'duration': 1.522}, {'end': 1748.678, 'text': 'So, first of all, any data that is stored in a relational database right in an which involves OLDB,', 'start': 1742.609, 'duration': 6.069}, {'end': 1751.741, 'text': 'that contains the current data as well as past data.', 'start': 1748.678, 'duration': 3.063}, {'end': 1757.648, 'text': 'okay, current data as well as positive, but with respect to a data warehouse and while performing a normal app,', 'start': 1751.741, 'duration': 5.907}, {'end': 1760.411, 'text': 'you will be dealing with only historical data here.', 'start': 1757.648, 'duration': 2.763}, {'end': 1765.119, 'text': 'okay, It contains only historical data and the data that will be stored in your database.', 'start': 1760.411, 'duration': 4.708}, {'end': 1769.383, 'text': 'Okay, when you use the OLTP, then those queries will be useful in running your business.', 'start': 1765.119, 'duration': 4.264}, {'end': 1775.269, 'text': 'Okay, when you have to run your business, like if you want to store the data of the number of sales that has happened today,', 'start': 1769.383, 'duration': 5.886}, {'end': 1779.453, 'text': 'Like every time a sales happens, then your records in your database has to be updated right?', 'start': 1775.269, 'duration': 4.184}], 'summary': 'Importance of data warehouse and key terminologies explained in a training session.', 'duration': 120.114, 'max_score': 1659.339, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81659339.jpg'}, {'end': 1748.678, 'src': 'embed', 'start': 1722.068, 'weight': 1, 'content': [{'end': 1727.454, 'text': "if you're running any kind of queries on your database, then that's called online transaction processing, okay?", 'start': 1722.068, 'duration': 5.386}, {'end': 1731.057, 'text': 'And then OLAP stands for online analytical processing.', 'start': 1727.914, 'duration': 3.143}, {'end': 1733.38, 'text': 'And this is the property of a data warehouse.', 'start': 1731.298, 'duration': 2.082}, {'end': 1740.107, 'text': "So any kind of query or any kind of analysis that you run on your data warehouse, that's called as an OLAP activity, correct?", 'start': 1733.88, 'duration': 6.227}, {'end': 1742.349, 'text': "So let's go to the differences between the two.", 'start': 1740.827, 'duration': 1.522}, {'end': 1748.678, 'text': 'So, first of all, any data that is stored in a relational database right in an which involves OLDB,', 'start': 1742.609, 'duration': 6.069}], 'summary': 'Oltp involves running queries on a database, while olap is for data warehouse analysis.', 'duration': 26.61, 'max_score': 1722.068, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81722068.jpg'}, {'end': 1929.84, 'src': 'embed', 'start': 1899.97, 'weight': 2, 'content': [{'end': 1903.512, 'text': 'So you basically use it for writing data into the database.', 'start': 1899.97, 'duration': 3.542}, {'end': 1907.974, 'text': 'But your data warehouse is primarily used for reading data from the data warehouse.', 'start': 1903.992, 'duration': 3.982}, {'end': 1913.237, 'text': 'So writing to the data warehouse is something that is done so that you can do the reading from the data warehouse.', 'start': 1908.314, 'duration': 4.923}, {'end': 1919.32, 'text': 'The primary concept here is to read the data from the data warehouse and to do the analysis and all the visualization activities.', 'start': 1913.257, 'duration': 6.063}, {'end': 1923.515, 'text': "But with the database, it's more of writing the data into the database.", 'start': 1919.912, 'duration': 3.603}, {'end': 1925.476, 'text': 'all right, and The size.', 'start': 1923.515, 'duration': 1.961}, {'end': 1929.84, 'text': 'so, speaking of the size, our databases size would raise anywhere between 100 MB to 1 GB.', 'start': 1925.476, 'duration': 4.364}], 'summary': 'Data warehouse for reading, database for writing. database size: 100mb to 1gb.', 'duration': 29.87, 'max_score': 1899.97, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81899970.jpg'}, {'end': 1971.603, 'src': 'embed', 'start': 1939.487, 'weight': 3, 'content': [{'end': 1940.628, 'text': 'if you look at a data warehouse,', 'start': 1939.487, 'duration': 1.141}, {'end': 1950.54, 'text': 'a data that our size ranges from 100 GB to 1 TB and Correct so your data values will have all the historic data and it will have all the relationships between the different data Right,', 'start': 1940.628, 'duration': 9.912}, {'end': 1952.883, 'text': 'such that you can do your analysis straight away.', 'start': 1950.54, 'duration': 2.343}, {'end': 1959.172, 'text': 'So, since it makes all the data more efficient and stuff the data here, the size ranges from 100 GB to almost 1 TB.', 'start': 1952.883, 'duration': 6.289}, {'end': 1964, 'text': "So that is what a data warehouse is, and that's the part of a data warehouse.", 'start': 1960.339, 'duration': 3.661}, {'end': 1967.742, 'text': "And I can actually show you the difference between the two in today's demo session.", 'start': 1964.561, 'duration': 3.181}, {'end': 1971.603, 'text': "Later during the session, I'll show you the size of the source file that I have.", 'start': 1967.982, 'duration': 3.621}], 'summary': 'A data warehouse can store 100 gb to 1 tb of data, enabling efficient analysis and historical data storage.', 'duration': 32.116, 'max_score': 1939.487, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81939487.jpg'}, {'end': 2028.063, 'src': 'embed', 'start': 1996.371, 'weight': 4, 'content': [{'end': 1996.912, 'text': 'All right.', 'start': 1996.371, 'duration': 0.541}, {'end': 1997.932, 'text': 'so your data values.', 'start': 1996.912, 'duration': 1.02}, {'end': 2001.074, 'text': "of course it's not as fast as your database, but it's highly flexible.", 'start': 1997.932, 'duration': 3.142}, {'end': 2003.835, 'text': "It's highly flexible because it gives you different views.", 'start': 2001.074, 'duration': 2.761}, {'end': 2006.076, 'text': 'So you have something called as the OLAP cube, right?', 'start': 2004.215, 'duration': 1.861}, {'end': 2009.478, 'text': 'So using the OLAP cube you can get the.', 'start': 2006.496, 'duration': 2.982}, {'end': 2014.48, 'text': 'you can look at insights from different angles, different perspectives and different views of data you will get.', 'start': 2009.478, 'duration': 5.002}, {'end': 2016.381, 'text': 'so that is the big advantage here.', 'start': 2014.48, 'duration': 1.901}, {'end': 2023.599, 'text': "and Okay, and the number of records that is accessed it isn't tens, but whereas with the data warehouse the number of records accessed isn't millions.", 'start': 2016.381, 'duration': 7.218}, {'end': 2028.063, 'text': 'All right, an example of this can be all the bank transactions made by a particular customer.', 'start': 2023.599, 'duration': 4.464}], 'summary': 'Olap cube provides flexible data views, accessing millions of records, beneficial for analyzing bank transactions.', 'duration': 31.692, 'max_score': 1996.371, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81996371.jpg'}, {'end': 2113.882, 'src': 'embed', 'start': 2092.96, 'weight': 5, 'content': [{'end': 2102.226, 'text': 'So if you want more examples of an OLTP, then one would be that of a supermarket server which records every single product purchased at that market.', 'start': 2092.96, 'duration': 9.266}, {'end': 2109.679, 'text': 'okay, so every single product in their history, or probably in the last one month, all these things can be accessed using your OLTP.', 'start': 2102.833, 'duration': 6.846}, {'end': 2111.941, 'text': 'okay, from your database.', 'start': 2109.679, 'duration': 2.262}, {'end': 2113.882, 'text': "you don't have options to do much of filtering here.", 'start': 2111.941, 'duration': 1.941}], 'summary': 'Oltp systems like a supermarket server record every product purchased, allowing access to extensive data from the database.', 'duration': 20.922, 'max_score': 2092.96, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM82092960.jpg'}], 'start': 1503.444, 'title': 'Data warehousing', 'summary': 'Discusses key properties and terminologies of data warehousing, including subject-oriented nature, integration of different data sources, and the differences between oltp and olap. it highlights the importance and benefits of data warehouses in providing integrated, historical, and analyzed data for business operations and analysis. additionally, it compares data warehouses and databases, emphasizing the differences in size, storage capabilities, query flexibility, and providing examples of oltp and olap strategies.', 'chapters': [{'end': 1919.32, 'start': 1503.444, 'title': 'Data warehousing overview', 'summary': 'Discusses the key properties and terminologies of data warehousing, including the subject-oriented nature, integration of different data sources, and the differences between oltp and olap, highlighting the importance and benefits of a data warehouse in providing integrated, historical, and analyzed data for business operations and analysis.', 'duration': 415.876, 'highlights': ['The data warehouse stores data by business subject, enabling integrated and single-view access to data from different teams and functions, such as sales, operations, and marketing in a retail company. The data warehouse categorizes and stores data by business subject, allowing integrated access to data from different teams, facilitating single-view data retrieval and analysis.', 'Data warehouse provides historical data and allows for OLAP activities, while OLTP focuses on current data and is based on the entity relationship model, emphasizing the differences in data storage and usage between the two systems. Data warehouse focuses on historical data and OLAP activities, while OLTP deals with current data and is based on the entity relationship model, showcasing the differences in data usage and storage between the two systems.', 'OLTP is used for writing data into the database, while the data warehouse is primarily used for reading data from the data warehouse, emphasizing their respective roles in data management and analysis. OLTP is utilized for writing data into the database, whereas the data warehouse is primarily used for reading and analyzing data, highlighting their distinct roles in data management.']}, {'end': 2206.445, 'start': 1919.912, 'title': 'Data warehouse vs database', 'summary': 'Discusses the differences between data warehouses and databases, highlighting that data warehouse sizes range from 100 gb to 1 tb, with the capability to store historic data and relationships, providing more detailed and accurate queries compared to databases, which are faster but less flexible, and examples of oltp and olap strategies are provided.', 'duration': 286.533, 'highlights': ['Data warehouse sizes range from 100 GB to 1 TB, storing historic data and relationships, providing more detailed and accurate queries. Data warehouse sizes and capabilities', 'Databases are faster but less flexible, while data warehouses are highly flexible and provide different views, such as OLAP cube, enabling insights from different angles and perspectives. Flexibility and views provided by data warehouses', 'Examples of OLTP include supermarket servers recording every single product purchased and bank servers recording every transaction, while OLAP provides more detailed and accurate answers, such as determining ATM usage and evaluating agent performance. Examples of OLTP and OLAP strategies']}], 'duration': 703.001, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM81503444.jpg', 'highlights': ['The data warehouse stores data by business subject, enabling integrated and single-view access to data from different teams and functions.', 'Data warehouse provides historical data and allows for OLAP activities, while OLTP focuses on current data and is based on the entity relationship model.', 'OLTP is used for writing data into the database, while the data warehouse is primarily used for reading data from the data warehouse.', 'Data warehouse sizes range from 100 GB to 1 TB, storing historic data and relationships, providing more detailed and accurate queries.', 'Databases are faster but less flexible, while data warehouses are highly flexible and provide different views, such as OLAP cube, enabling insights from different angles and perspectives.', 'Examples of OLTP include supermarket servers recording every single product purchased and bank servers recording every transaction, while OLAP provides more detailed and accurate answers, such as determining ATM usage and evaluating agent performance.']}, {'end': 3285.549, 'segs': [{'end': 2313.06, 'src': 'heatmap', 'start': 2250.353, 'weight': 0.932, 'content': [{'end': 2255.118, 'text': "You're extracting the data from here, you're extracting it, and then you're transforming it into the way you want.", 'start': 2250.353, 'duration': 4.765}, {'end': 2258.282, 'text': 'in a more readable fashion, in a more relatable fashion.', 'start': 2255.698, 'duration': 2.584}, {'end': 2261.987, 'text': 'Then once that is done, you load that data into a data warehouse.', 'start': 2258.863, 'duration': 3.124}, {'end': 2266.955, 'text': 'And the whole process of getting the data from here data source to your data warehouse.', 'start': 2262.448, 'duration': 4.507}, {'end': 2272.102, 'text': 'this is done by the ETL the activities of extraction, transformation and loading.', 'start': 2266.955, 'duration': 5.147}, {'end': 2275.516, 'text': 'So we have popular tools for this very process.', 'start': 2272.455, 'duration': 3.061}, {'end': 2280.358, 'text': 'So you have tools like Talend, Informatica, you have Erwin, all these things.', 'start': 2275.917, 'duration': 4.441}, {'end': 2288.282, 'text': 'And Informatica and Talend are probably the most popular tools for this process for extraction, transformation and loading data into a data warehouse.', 'start': 2280.759, 'duration': 7.523}, {'end': 2291.964, 'text': 'So this is something that you should have understood by now.', 'start': 2288.962, 'duration': 3.002}, {'end': 2294.485, 'text': "Any doubts, guys? Because I don't want to waste much time.", 'start': 2292.404, 'duration': 2.081}, {'end': 2297.146, 'text': 'I want to go to the next slide and tease the next concept.', 'start': 2294.525, 'duration': 2.621}, {'end': 2298.913, 'text': 'okay, great.', 'start': 2297.952, 'duration': 0.961}, {'end': 2301.274, 'text': 'so the next one is data Mart.', 'start': 2298.913, 'duration': 2.361}, {'end': 2305.576, 'text': "okay, now, if you've understood so far, till ETL, then half your job is done,", 'start': 2301.274, 'duration': 4.302}, {'end': 2313.06, 'text': "because data Mart is something that's very close to a data warehouse and you don't have that much of a difference when it compared to a data warehouse.", 'start': 2305.576, 'duration': 7.484}], 'summary': 'Etl process involves extracting, transforming, and loading data into a data warehouse using tools like talend and informatica.', 'duration': 62.707, 'max_score': 2250.353, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM82250353.jpg'}, {'end': 2288.282, 'src': 'embed', 'start': 2262.448, 'weight': 1, 'content': [{'end': 2266.955, 'text': 'And the whole process of getting the data from here data source to your data warehouse.', 'start': 2262.448, 'duration': 4.507}, {'end': 2272.102, 'text': 'this is done by the ETL the activities of extraction, transformation and loading.', 'start': 2266.955, 'duration': 5.147}, {'end': 2275.516, 'text': 'So we have popular tools for this very process.', 'start': 2272.455, 'duration': 3.061}, {'end': 2280.358, 'text': 'So you have tools like Talend, Informatica, you have Erwin, all these things.', 'start': 2275.917, 'duration': 4.441}, {'end': 2288.282, 'text': 'And Informatica and Talend are probably the most popular tools for this process for extraction, transformation and loading data into a data warehouse.', 'start': 2280.759, 'duration': 7.523}], 'summary': 'Etl process involves popular tools like talend and informatica for data warehouse.', 'duration': 25.834, 'max_score': 2262.448, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM82262448.jpg'}, {'end': 2348.165, 'src': 'embed', 'start': 2320.083, 'weight': 2, 'content': [{'end': 2322.905, 'text': "okay. so let's look at the line, the definition here.", 'start': 2320.083, 'duration': 2.822}, {'end': 2326.927, 'text': 'the data Mart is a smaller version of the data warehouse which deals with a single subject.', 'start': 2322.905, 'duration': 4.022}, {'end': 2330.098, 'text': 'Okay, the data much are focused on one area.', 'start': 2327.617, 'duration': 2.481}, {'end': 2338.481, 'text': 'Hence. They draw the data from limited number of sources and the time taken to build data much is very less compared to the time taken to build a data warehouse.', 'start': 2330.398, 'duration': 8.083}, {'end': 2346.344, 'text': 'now, to give you an example or an explanation of this, in your data warehouse you will have all your details right, all your details that you have.', 'start': 2338.481, 'duration': 7.863}, {'end': 2348.165, 'text': 'So this itself is more refined.', 'start': 2346.424, 'duration': 1.741}], 'summary': 'Data mart is a smaller version of a data warehouse, focusing on a single subject, drawing data from limited sources, and quicker to build.', 'duration': 28.082, 'max_score': 2320.083, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM82320083.jpg'}, {'end': 2657.275, 'src': 'embed', 'start': 2630.304, 'weight': 7, 'content': [{'end': 2636.886, 'text': 'Your dependent data mart is the data is first extracted from the OLTP systems and then populated in the central data warehouse.', 'start': 2630.304, 'duration': 6.582}, {'end': 2640.386, 'text': 'And then from this data warehouse, the data travels to the data mart.', 'start': 2637.266, 'duration': 3.12}, {'end': 2641.327, 'text': 'So look at this example.', 'start': 2640.406, 'duration': 0.921}, {'end': 2646.408, 'text': 'This is the standard practice or the default approach where you have an OLTP source.', 'start': 2641.567, 'duration': 4.841}, {'end': 2648.608, 'text': 'Then you get that data into a data warehouse.', 'start': 2646.828, 'duration': 1.78}, {'end': 2650.829, 'text': 'And then from the data warehouse, you form a data mart.', 'start': 2648.848, 'duration': 1.981}, {'end': 2657.275, 'text': "Okay, we'll have multiple data marks, where each different mark will have particular data from the entire data warehouse.", 'start': 2651.189, 'duration': 6.086}], 'summary': 'Data is extracted from oltp systems to populate the data warehouse, which then feeds into multiple data marts.', 'duration': 26.971, 'max_score': 2630.304, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM82630304.jpg'}, {'end': 2790.552, 'src': 'heatmap', 'start': 2725.928, 'weight': 0.779, 'content': [{'end': 2731.052, 'text': 'so this is what the hybrid data mart is and depending on your company, depending on the size of your company,', 'start': 2725.928, 'duration': 5.124}, {'end': 2735.068, 'text': 'the requirements of your company or your organization, you can choose one of these.', 'start': 2731.567, 'duration': 3.501}, {'end': 2740.269, 'text': 'You can model your entire database and data warehouse in any one of these models,', 'start': 2735.508, 'duration': 4.761}, {'end': 2743.909, 'text': 'either the dependent or the independent data match or hybrid data match.', 'start': 2740.269, 'duration': 3.64}, {'end': 2744.189, 'text': 'all right?', 'start': 2743.909, 'duration': 0.28}, {'end': 2746.65, 'text': "So that's about the different types of data match.", 'start': 2744.689, 'duration': 1.961}, {'end': 2749.75, 'text': 'So moving on to the next slide, we have something called as metadata.', 'start': 2747.19, 'duration': 2.56}, {'end': 2756.271, 'text': 'Now people here from programming background or from the technology background, you might all be aware of what a metadata is.', 'start': 2750.59, 'duration': 5.681}, {'end': 2764.467, 'text': 'Metadata basically is defined as data about data, okay? It contains data about where your actual data is stored.', 'start': 2756.571, 'duration': 7.896}, {'end': 2766.809, 'text': 'Supposing you have your raw data right?', 'start': 2764.767, 'duration': 2.042}, {'end': 2768.47, 'text': 'Where is that data stored??', 'start': 2767.289, 'duration': 1.181}, {'end': 2770.111, 'text': 'What is the size of that data??', 'start': 2768.55, 'duration': 1.561}, {'end': 2773.194, 'text': 'So these are the answers to these kind of questions.', 'start': 2770.552, 'duration': 2.642}, {'end': 2774.735, 'text': 'is what will be present in your metadata?', 'start': 2773.194, 'duration': 1.541}, {'end': 2778.222, 'text': 'Your metadata will contain the location of your actual data.', 'start': 2775.219, 'duration': 3.003}, {'end': 2780.063, 'text': 'It will contain the size of your actual data.', 'start': 2778.442, 'duration': 1.621}, {'end': 2785.248, 'text': 'It will contain details like which was the source it came from and when was it created.', 'start': 2780.504, 'duration': 4.744}, {'end': 2787.93, 'text': 'All these details will be stored in your metadata.', 'start': 2785.708, 'duration': 2.222}, {'end': 2790.552, 'text': "So that's how different metadata is from regular data.", 'start': 2788.19, 'duration': 2.362}], 'summary': 'Hybrid data mart options and metadata details were discussed in the presentation.', 'duration': 64.624, 'max_score': 2725.928, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM82725928.jpg'}, {'end': 2826.621, 'src': 'embed', 'start': 2794.002, 'weight': 0, 'content': [{'end': 2797.844, 'text': 'And a metadata is specifically in a data warehouse, it defines the source data.', 'start': 2794.002, 'duration': 3.842}, {'end': 2801.046, 'text': 'There is a flat file relational database and other objects.', 'start': 2797.864, 'duration': 3.182}, {'end': 2810.326, 'text': "So the reason that we give so much importance to metadata in a data warehouse is because Take the example of any company that's having a 24-7 business.", 'start': 2801.447, 'duration': 8.879}, {'end': 2813.849, 'text': 'They have a rolling sales team that works throughout the clock.', 'start': 2810.927, 'duration': 2.922}, {'end': 2818.354, 'text': 'The 24-7 they have sales coming in, data will be going into your database.', 'start': 2814.77, 'duration': 3.584}, {'end': 2826.621, 'text': 'Now in this case, you cannot always keep adding data into your data warehouse because you know that data warehouse is not real time.', 'start': 2818.714, 'duration': 7.907}], 'summary': 'Metadata in data warehouse is crucial due to 24-7 business, requiring efficient data management and storage.', 'duration': 32.619, 'max_score': 2794.002, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM82794002.jpg'}, {'end': 3088.359, 'src': 'heatmap', 'start': 3014.401, 'weight': 3, 'content': [{'end': 3016.262, 'text': "so that's the thing about metadata.", 'start': 3014.401, 'duration': 1.861}, {'end': 3019.483, 'text': 'guys, now going to the next slide, we have the architecture.', 'start': 3016.262, 'duration': 3.221}, {'end': 3025.486, 'text': 'so now that you know everything, okay, these four terminologies are more than enough for you to understand the architecture.', 'start': 3019.483, 'duration': 6.003}, {'end': 3027.947, 'text': "okay, so let's understand what the architecture is.", 'start': 3025.486, 'duration': 2.461}, {'end': 3030.428, 'text': 'so this is the entire architecture diagram.', 'start': 3027.947, 'duration': 2.481}, {'end': 3033.73, 'text': 'okay, So this is what data comes in from various sources.', 'start': 3030.428, 'duration': 3.302}, {'end': 3038.393, 'text': 'It can come from either a database or it can come from flat file and then that data,', 'start': 3033.81, 'duration': 4.583}, {'end': 3042.837, 'text': 'an action of ETL will be performed on that data and it will go to the staging area.', 'start': 3038.393, 'duration': 4.444}, {'end': 3049.102, 'text': 'This is called the staging area and this is the staging database and the data that is stored over here, it is temporary data.', 'start': 3042.997, 'duration': 6.105}, {'end': 3055.867, 'text': 'Before the data completely moves to the data warehouse, it will be present in this area and that is done by using the act of ETL.', 'start': 3049.542, 'duration': 6.325}, {'end': 3059.889, 'text': 'and also between moving to the data warehouse.', 'start': 3056.707, 'duration': 3.182}, {'end': 3061.751, 'text': 'the process of ETL continues.', 'start': 3059.889, 'duration': 1.862}, {'end': 3067.315, 'text': 'so ETL process starts over here and it ends over here, okay, and between the conversion,', 'start': 3061.751, 'duration': 5.564}, {'end': 3073.339, 'text': 'it is temporarily stored in a staging area and this is most often present inside the ETL tool itself.', 'start': 3067.315, 'duration': 6.024}, {'end': 3076.741, 'text': 'okay, like your talent or informatica and all these things.', 'start': 3073.339, 'duration': 3.402}, {'end': 3079.243, 'text': 'and then this data will be stored in your data warehouse.', 'start': 3076.741, 'duration': 2.502}, {'end': 3086.598, 'text': 'so whatever is extracted, transformed and loaded, it will be loaded into your data warehouse and in your data warehouse you will have metadata, okay,', 'start': 3079.243, 'duration': 7.355}, {'end': 3088.359, 'text': "and of course you'll have your raw data.", 'start': 3086.598, 'duration': 1.761}], 'summary': 'Metadata, architecture, and etl process explained for data warehouse with staging area.', 'duration': 73.958, 'max_score': 3014.401, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83014401.jpg'}, {'end': 3119.258, 'src': 'embed', 'start': 3086.598, 'weight': 4, 'content': [{'end': 3088.359, 'text': "and of course you'll have your raw data.", 'start': 3086.598, 'duration': 1.761}, {'end': 3090.04, 'text': 'and then you will have your aggregate data.', 'start': 3088.359, 'duration': 1.681}, {'end': 3099.286, 'text': "okay, and this is the reason why a data warehouse is generally you know, it's larger in size, because it has not just raw data, your database, which,", 'start': 3090.04, 'duration': 9.246}, {'end': 3101.767, 'text': 'from where the data is coming in, it will only have your raw data.', 'start': 3099.286, 'duration': 2.481}, {'end': 3104.829, 'text': 'okay, but your data warehouse will have additional stuff here.', 'start': 3101.767, 'duration': 3.062}, {'end': 3107.231, 'text': "it'll have also your metadata and your aggregate data.", 'start': 3104.829, 'duration': 2.402}, {'end': 3110.813, 'text': 'and together, all these three things together, is what helping you?', 'start': 3107.231, 'duration': 3.582}, {'end': 3119.258, 'text': 'being you, you doing your analysis sooner, okay, this is what powers your library to do OLAP, online analytical processing, okay.', 'start': 3110.813, 'duration': 8.445}], 'summary': 'Data warehouse holds raw, aggregate, and metadata, enabling quicker olap analysis.', 'duration': 32.66, 'max_score': 3086.598, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83086598.jpg'}, {'end': 3190.44, 'src': 'embed', 'start': 3170.454, 'weight': 5, 'content': [{'end': 3180.597, 'text': 'because your data warehouse will be divided into different data marts and your data your different groups we get access to only that data which they want to or which they can get access to correct?', 'start': 3170.454, 'duration': 10.143}, {'end': 3186.978, 'text': "So this way no group can get access to every data that is present in a data warehouse and there's advanced.", 'start': 3180.917, 'duration': 6.061}, {'end': 3188.859, 'text': "there's a little more data security in this case.", 'start': 3186.978, 'duration': 1.881}, {'end': 3190.44, 'text': 'Okay, same thing over here.', 'start': 3189.239, 'duration': 1.201}], 'summary': 'Data warehouse divided into data marts for controlled access, enhancing data security.', 'duration': 19.986, 'max_score': 3170.454, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83170454.jpg'}, {'end': 3247.137, 'src': 'embed', 'start': 3224.802, 'weight': 6, 'content': [{'end': 3234.191, 'text': "I'll be getting it from my Oracle database and I'll be storing that into a data warehouse which can be ready for any kind of analysis or visualization on any other visualization tool.", 'start': 3224.802, 'duration': 9.389}, {'end': 3240.693, 'text': "So this act over here, which I'm gonna show you this, is what powers your business intelligence, right?", 'start': 3235.29, 'duration': 5.403}, {'end': 3242.754, 'text': 'So are you guys ready for the demonstration?', 'start': 3241.233, 'duration': 1.521}, {'end': 3243.835, 'text': 'Any doubts here, guys?', 'start': 3243.014, 'duration': 0.821}, {'end': 3245.135, 'text': 'Okay great.', 'start': 3244.655, 'duration': 0.48}, {'end': 3247.137, 'text': "Rodney says he's all ready, he's all pumped up.", 'start': 3245.656, 'duration': 1.481}], 'summary': 'Data from oracle database stored in data warehouse for analysis and visualization, powering business intelligence.', 'duration': 22.335, 'max_score': 3224.802, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83224802.jpg'}], 'start': 2206.445, 'title': 'Data warehouse, etl, data marts, and metadata', 'summary': 'Covers key concepts of data warehouse, including olap, etl, and data mart, highlighting differences, purposes, and types. it also explains the etl process, staging area, and data warehouse components, emphasizing the significance of metadata in data warehousing and the types of data marts, including independent, dependent, and hybrid.', 'chapters': [{'end': 2682.276, 'start': 2206.445, 'title': 'Data warehouse and etl process', 'summary': 'Discusses the key concepts of data warehouse, including olap, etl, and data mart, highlighting the differences, purposes, and types, emphasizing the significance of etl in data extraction, transformation, and loading, and the advantages of using data marts for specific user bases.', 'duration': 475.831, 'highlights': ['The ETL process involves extracting data from various sources, transforming it to meet requirements, and loading it into a target data warehouse, and popular tools for this process include Talend, Informatica, and Erwin.', 'Data Marts are smaller versions of data warehouses, focusing on single subjects, drawing data from limited sources, and taking less time to build compared to data warehouses, serving specific user bases like sales, marketing, and operations teams.', 'Data Warehouses store enterprise-wide data, have multiple subject areas, and occupy large memory, while Data Marts store department-wise data, have a single subject area, limited data sources, and occupy limited memory, taking a shorter time to implement.', 'There are three types of Data Marts: dependent, independent, and hybrid, with dependent data marts following the standard practice of data extraction from OLTP systems to a central data warehouse and then to the data mart.']}, {'end': 3033.73, 'start': 2682.276, 'title': 'Types of data marts and the significance of metadata in data warehousing', 'summary': 'Explains the different types of data marts, including independent, dependent, and hybrid, and emphasizes the significance of metadata in data warehousing, highlighting its role in defining source data, saving time, and enabling business logic transformation.', 'duration': 351.454, 'highlights': ['The chapter explains the different types of data marts, including independent, dependent, and hybrid The data marts are categorized as independent, dependent, and hybrid, depending on the source and usage, and multiple data marts can be used in a company.', 'The significance of metadata in data warehousing, highlighting its role in defining source data, saving time, and enabling business logic transformation Metadata is crucial in data warehousing as it defines the source data, saves time by automating data import processes, and facilitates business logic transformation, making it the epitome of data warehousing.', 'The architecture diagram represents the flow of data from various sources into the data warehouse The architecture diagram illustrates the flow of data from various sources into the data warehouse, providing a visual representation of the data warehousing process.']}, {'end': 3285.549, 'start': 3033.81, 'title': 'Etl process and data warehousing', 'summary': 'Explains the etl process, staging area, data warehouse components, data marts, and demonstrates using talent bi for importing data into a data warehouse.', 'duration': 251.739, 'highlights': ['The ETL process involves extracting data from a source (database or flat file), transforming it, and loading it into the staging area before moving to the data warehouse. The ETL process involves extracting data from a source, transforming it, and loading it into the staging area before moving to the data warehouse. This temporary storage in the staging area aids in the ETL process and is often present inside ETL tools like Talend or Informatica.', 'Data warehouse contains raw data, metadata, and aggregate data, enabling OLAP for analysis, making it larger than a regular database. The data warehouse contains raw data, metadata, and aggregate data, enabling OLAP for analysis, making it larger than a regular database.', 'Data warehouse can be divided into data marts for different user groups to access specific data, enhancing data security and access control. Data warehouse can be divided into data marts for different user groups to access specific data, enhancing data security and access control, ensuring that each group can access only the data they need.', 'The demonstration using talent BI illustrates importing data from an Oracle database to create a data warehouse for business intelligence purposes. The demonstration using talent BI illustrates importing data from an Oracle database to create a data warehouse for business intelligence purposes, highlighting the practical application of the concepts explained.']}], 'duration': 1079.104, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM82206445.jpg', 'highlights': ['Metadata is crucial in data warehousing as it defines the source data, saves time by automating data import processes, and facilitates business logic transformation, making it the epitome of data warehousing.', 'The ETL process involves extracting data from various sources, transforming it to meet requirements, and loading it into a target data warehouse, and popular tools for this process include Talend, Informatica, and Erwin.', 'Data Marts are smaller versions of data warehouses, focusing on single subjects, drawing data from limited sources, and taking less time to build compared to data warehouses, serving specific user bases like sales, marketing, and operations teams.', 'The architecture diagram illustrates the flow of data from various sources into the data warehouse, providing a visual representation of the data warehousing process.', 'Data warehouse contains raw data, metadata, and aggregate data, enabling OLAP for analysis, making it larger than a regular database.', 'Data warehouse can be divided into data marts for different user groups to access specific data, enhancing data security and access control, ensuring that each group can access only the data they need.', 'The demonstration using talent BI illustrates importing data from an Oracle database to create a data warehouse for business intelligence purposes, highlighting the practical application of the concepts explained.', 'There are three types of Data Marts: dependent, independent, and hybrid, with dependent data marts following the standard practice of data extraction from OLTP systems to a central data warehouse and then to the data mart.']}, {'end': 3665.786, 'segs': [{'end': 3311.685, 'src': 'embed', 'start': 3285.889, 'weight': 0, 'content': [{'end': 3292.073, 'text': "Okay, now the data set that I'll be using is that of a 10,000 row table and a 3,000 row table.", 'start': 3285.889, 'duration': 6.184}, {'end': 3295.816, 'text': "Okay, so there'll be one table having 10,000 customer details.", 'start': 3292.233, 'duration': 3.583}, {'end': 3302.418, 'text': "Okay. and then there'll be another table having 50,000 rows of transactions which either those customers make.", 'start': 3296.533, 'duration': 5.885}, {'end': 3307.882, 'text': 'okay. now, based on this data, We have to find those customers who have the lowest number of purchases, okay?', 'start': 3302.418, 'duration': 5.464}, {'end': 3311.685, 'text': 'So right now my data set is present in my Oracle database.', 'start': 3308.222, 'duration': 3.463}], 'summary': 'Analyzing 10,000 customer details and 50,000 transactions to find lowest purchasing customers.', 'duration': 25.796, 'max_score': 3285.889, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83285889.jpg'}, {'end': 3427.382, 'src': 'embed', 'start': 3390.585, 'weight': 1, 'content': [{'end': 3397.709, 'text': 'Since these are two different tables, I can create my data warehouse with the help of this particular primary and foreign key concept.', 'start': 3390.585, 'duration': 7.124}, {'end': 3399.25, 'text': 'I can create a link and I can create.', 'start': 3397.709, 'duration': 1.541}, {'end': 3404.092, 'text': 'I can probably just link these two tables with the primary key, foreign key concept.', 'start': 3399.25, 'duration': 4.842}, {'end': 3406.493, 'text': 'okay, so Any doubts, guys?', 'start': 3404.092, 'duration': 2.401}, {'end': 3413.397, 'text': "I'm sure that you all know what a primary key and a foreign key is and if at all anybody has a problem, Please let me know.", 'start': 3406.653, 'duration': 6.744}, {'end': 3416.92, 'text': "Okay, So Rodney says I don't know what's a foreign key.", 'start': 3413.397, 'duration': 3.523}, {'end': 3421.741, 'text': 'So see, Rodney, the thing is, we have something called as a primary key and a foreign key.', 'start': 3417.62, 'duration': 4.121}, {'end': 3427.382, 'text': 'And we use these two columns when we want to use a combination of two different tables.', 'start': 3422.181, 'duration': 5.201}], 'summary': 'Creating data warehouse using primary and foreign key concept for linking tables.', 'duration': 36.797, 'max_score': 3390.585, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83390585.jpg'}, {'end': 3539.62, 'src': 'embed', 'start': 3512.577, 'weight': 3, 'content': [{'end': 3516.461, 'text': "And what we're gonna do is for every customer ID over here, Right.", 'start': 3512.577, 'duration': 3.884}, {'end': 3522.565, 'text': "so for every customer ID over here, we're gonna link it to this particular table with the help of this customer ID.", 'start': 3516.461, 'duration': 6.104}, {'end': 3525.827, 'text': 'Since the customer ID is a common field in both the cases,', 'start': 3523.066, 'duration': 2.761}, {'end': 3534.473, 'text': 'I can make a table which would show which customer which would probably display all these details along with the other columns.', 'start': 3525.827, 'duration': 8.646}, {'end': 3539.62, 'text': "okay, now that's because i have customer id, which is common in both the tables,", 'start': 3535.017, 'duration': 4.603}], 'summary': 'Link customer ids to display details in a common table.', 'duration': 27.043, 'max_score': 3512.577, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83512577.jpg'}, {'end': 3618.457, 'src': 'embed', 'start': 3589.127, 'weight': 4, 'content': [{'end': 3593.591, 'text': "I didn't want these details because I was not gonna use them in my data warehouse.", 'start': 3589.127, 'duration': 4.464}, {'end': 3599.875, 'text': 'So if you see our problem statement here, it only says you have to find out the customers who have the lower number of purchases.', 'start': 3593.871, 'duration': 6.004}, {'end': 3605.22, 'text': "It doesn't mention that you need their country details or city details or anything for that matter of fact.", 'start': 3600.196, 'duration': 5.024}, {'end': 3611.195, 'text': 'So for that reason, using the country, city, and address fields would be a waste.', 'start': 3606.533, 'duration': 4.662}, {'end': 3613.316, 'text': "So that's why I've not imported these into my data warehouse.", 'start': 3611.275, 'duration': 2.041}, {'end': 3618.457, 'text': 'I have only the customer ID, the customer name, my contact number, and my email address fields.', 'start': 3613.776, 'duration': 4.681}], 'summary': 'Data warehouse excludes country, city, and address fields to focus on customer id, name, contact number, and email address.', 'duration': 29.33, 'max_score': 3589.127, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83589127.jpg'}], 'start': 3285.889, 'title': 'Analyzing customer purchase data and understanding primary and foreign keys in database tables', 'summary': 'Discusses analyzing a 10,000-row customer table and a 3,000-row transaction table to identify customers with the lowest number of purchases, using primary and foreign key concepts to link the two tables. it also covers the concept of primary and foreign keys, linking tables using a common field, and the design decisions made while importing data into a database for a specific problem statement, emphasizing the importance of relevant data for analysis.', 'chapters': [{'end': 3494.753, 'start': 3285.889, 'title': 'Analyzing customer purchase data', 'summary': 'Discusses analyzing a 10,000-row customer table and a 3,000-row transaction table to identify customers with the lowest number of purchases, using primary and foreign key concepts to link the two tables.', 'duration': 208.864, 'highlights': ['The dataset includes a 10,000-row customer table and a 3,000-row transaction table. The speaker mentions the specific size of the dataset, comprising 10,000 rows for the customer table and 3,000 rows for the transaction table.', 'Explaining the primary and foreign key concept for linking tables The speaker explains the primary key and foreign key concept, emphasizing the use of customer ID as the primary key in the customer table and as the foreign key in the transactions table to establish a link between the two tables.', "Clarifying the purpose of primary and foreign keys in linking tables The speaker elaborates on the significance of primary and foreign keys in linking the two different tables, emphasizing the uniqueness of customer ID as a primary key and demonstrating how it relates to the customer's transactions in the other table."]}, {'end': 3665.786, 'start': 3494.773, 'title': 'Understanding primary and foreign keys in database tables', 'summary': 'Discusses the concept of primary and foreign keys, linking tables using a common field, and the design decisions made while importing data into a database for a specific problem statement, emphasizing the importance of relevant data for analysis.', 'duration': 171.013, 'highlights': ['The uniqueness of primary and foreign keys, and their roles in ensuring data integrity, are explained. The speaker explains that the primary key ensures uniqueness and will not appear again, while the foreign key can appear more than once, providing a clear distinction between the two.', 'Linking tables using a common field and creating a table to display details is demonstrated. The speaker describes linking tables by using the common field of customer ID and creating a table to display details from both tables, emphasizing the connection between the two based on the common field.', 'Design decisions made while importing data into a database for a specific problem statement are discussed. The speaker explains the decision to ignore certain fields when importing an Excel file into the database, highlighting the necessity of relevant data for the specific problem statement to avoid importing unnecessary details.']}], 'duration': 379.897, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83285889.jpg', 'highlights': ['The dataset includes a 10,000-row customer table and a 3,000-row transaction table.', 'Explaining the primary and foreign key concept for linking tables.', 'The uniqueness of primary and foreign keys, and their roles in ensuring data integrity, are explained.', 'Linking tables using a common field and creating a table to display details is demonstrated.', 'Design decisions made while importing data into a database for a specific problem statement are discussed.']}, {'end': 4104.682, 'segs': [{'end': 3707.425, 'src': 'embed', 'start': 3665.786, 'weight': 0, 'content': [{'end': 3676.629, 'text': 'then we can extract the data such that the data warehouse gets the details of only those people who have made purchases of less than 10 items per order.', 'start': 3665.786, 'duration': 10.843}, {'end': 3680.69, 'text': 'So per transaction, whoever has made less than 10 purchases.', 'start': 3677.169, 'duration': 3.521}, {'end': 3681.95, 'text': 'So this person has made seven.', 'start': 3680.73, 'duration': 1.22}, {'end': 3684.294, 'text': 'and this person has made nine right?', 'start': 3682.512, 'duration': 1.782}, {'end': 3687.438, 'text': 'So we can get a list of those customers right?', 'start': 3684.695, 'duration': 2.743}, {'end': 3695.047, 'text': 'Using the customer ID, we can go to that customer table and get his name, we can get his email address, his phone number and other details okay?', 'start': 3687.578, 'duration': 7.469}, {'end': 3703.941, 'text': 'and we can publish all those things and we can probably export them to another CSV file like this, or we can put that into our database.', 'start': 3695.891, 'duration': 8.05}, {'end': 3707.425, 'text': 'Basically into anything which would support data visualization.', 'start': 3704.662, 'duration': 2.763}], 'summary': 'Extract data for customers with less than 10 purchases per order for data visualization.', 'duration': 41.639, 'max_score': 3665.786, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83665786.jpg'}, {'end': 3756.606, 'src': 'embed', 'start': 3727.562, 'weight': 2, 'content': [{'end': 3730.985, 'text': 'So first of all let me introduce you to our talent interface.', 'start': 3727.562, 'duration': 3.423}, {'end': 3737.369, 'text': 'So this was the Oracle SQL database which I showed you earlier and this one is the talent.', 'start': 3731.005, 'duration': 6.364}, {'end': 3738.83, 'text': 'So talent open studio.', 'start': 3737.829, 'duration': 1.001}, {'end': 3742.759, 'text': 'This is basically the data integration version of the project.', 'start': 3739.557, 'duration': 3.202}, {'end': 3747.301, 'text': "Now what I'm gonna do here is I'm gonna show you how talent works.", 'start': 3743.399, 'duration': 3.902}, {'end': 3748.922, 'text': "I'm gonna show you the layout over here first of all.", 'start': 3747.361, 'duration': 1.561}, {'end': 3752.744, 'text': 'So on the left here, you can see that you have something called as job design.', 'start': 3749.722, 'duration': 3.022}, {'end': 3756.606, 'text': 'We have a standard, you have demo, and then you have various other projects here.', 'start': 3753.124, 'duration': 3.482}], 'summary': 'Introducing talent interface and demonstrating its layout and functionality.', 'duration': 29.044, 'max_score': 3727.562, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83727562.jpg'}, {'end': 3854.375, 'src': 'embed', 'start': 3830.271, 'weight': 3, 'content': [{'end': 3837.218, 'text': 'So this is where I can get all the data from my different databases or fields, files, and get them to my data warehouse.', 'start': 3830.271, 'duration': 6.947}, {'end': 3839.541, 'text': 'So I can write my projects over here.', 'start': 3837.579, 'duration': 1.962}, {'end': 3841.803, 'text': 'And then, on the right hand side, you have your palette.', 'start': 3839.961, 'duration': 1.842}, {'end': 3847.329, 'text': 'okay?. So from your palette, you can choose the files or your database from where you wanna import your data.', 'start': 3841.803, 'duration': 5.526}, {'end': 3849.411, 'text': 'Okay, supposing you want to import from a flat file.', 'start': 3847.609, 'duration': 1.802}, {'end': 3852.614, 'text': 'Then you can just go to this heading file here.', 'start': 3849.891, 'duration': 2.723}, {'end': 3854.375, 'text': 'so under file, you have input right.', 'start': 3852.614, 'duration': 1.761}], 'summary': 'Data can be imported from different sources to the data warehouse using the palette and file options.', 'duration': 24.104, 'max_score': 3830.271, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83830271.jpg'}, {'end': 3896.105, 'src': 'embed', 'start': 3867.868, 'weight': 5, 'content': [{'end': 3873.353, 'text': 'and when you just enter further fields, you can get the data in from these excel files to your data warehouse.', 'start': 3867.868, 'duration': 5.485}, {'end': 3878.136, 'text': "Okay, to your talent, Okay, but excel file is a very simple thing and I don't want to show you that.", 'start': 3873.353, 'duration': 4.783}, {'end': 3882.158, 'text': 'let me show you how to import This large data from our database, right.', 'start': 3878.136, 'duration': 4.022}, {'end': 3886.82, 'text': 'so we have also other options here, like big data, big data, Intel, business intelligence.', 'start': 3882.158, 'duration': 4.662}, {'end': 3888.841, 'text': 'we have business, we have cloud, right.', 'start': 3886.82, 'duration': 2.021}, {'end': 3896.105, 'text': 'so we have integrations with a numerous technologies here, and What we would be concentrating now on is that of databases.', 'start': 3888.841, 'duration': 7.264}], 'summary': 'Demonstrating data import from various sources, focusing on databases.', 'duration': 28.237, 'max_score': 3867.868, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83867868.jpg'}, {'end': 3947.91, 'src': 'embed', 'start': 3925.288, 'weight': 4, 'content': [{'end': 3933.313, 'text': "So, first of all, I need to configure all these details with respect to my host, the port number on which it's hosted, the database name,", 'start': 3925.288, 'duration': 8.025}, {'end': 3935.774, 'text': 'the username, the password and all these things okay?', 'start': 3933.313, 'duration': 2.461}, {'end': 3939.576, 'text': 'So let me first start off from the host.', 'start': 3936.334, 'duration': 3.242}, {'end': 3942.158, 'text': "It's hosted on localhost.", 'start': 3940.417, 'duration': 1.741}, {'end': 3944.026, 'text': "It's not on any server.", 'start': 3942.805, 'duration': 1.221}, {'end': 3947.91, 'text': "It's not on any remote server, so I'm just gonna say localhost, and the port number is 1521.", 'start': 3944.467, 'duration': 3.443}], 'summary': 'Configuring details for localhost host and port 1521.', 'duration': 22.622, 'max_score': 3925.288, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83925288.jpg'}], 'start': 3665.786, 'title': 'Data extraction and talent interface', 'summary': 'Discusses data extraction to identify customers purchasing less than 10 items per order, obtaining their name, email, and phone for visualization. it also introduces talent interface for data integration, showcasing job design, data import, and configuration using gui and oracle sql database.', 'chapters': [{'end': 3726.902, 'start': 3665.786, 'title': 'Data extraction and visualization', 'summary': 'Discusses data extraction to identify customers who made purchases of less than 10 items per order, and using customer id to obtain their name, email address, and phone number for data visualization.', 'duration': 61.116, 'highlights': ['Data extraction to identify customers who made purchases of less than 10 items per order, allowing for targeted marketing and analysis.', 'Using customer ID to obtain customer details for data visualization, enabling personalized marketing strategies and customer relationship management.']}, {'end': 4104.682, 'start': 3727.562, 'title': 'Introduction to talent interface', 'summary': 'Introduces the talent interface for data integration, demonstrating how to create a new job design, import data from databases, and configure schema in talent open studio, emphasizing the use of gui and drag-and-drop functionality with oracle sql database. it provides insights into the various options and configuration details for importing data from different sources and mapping fields to the data warehouse.', 'duration': 377.12, 'highlights': ['The chapter introduces the talent interface for data integration and demonstrates the process of creating a new job design and importing data from databases. The speaker introduces Talent Open Studio as the data integration version of the project, emphasizing the use of GUI and drag-and-drop functionality to create jobs on the UI.', 'The demonstration focuses on the configuration details for importing data from the Oracle database, including host, port number, database name, username, and password. The speaker details the process of configuring the Oracle database input, specifying the host, port number, database name, username, and password, along with the schema mapping and field definitions.', 'Insights into the various options and configuration details for importing data from different sources and mapping fields to the data warehouse are provided. The chapter provides insights into the options for importing data from different sources, such as flat files and databases, and configuring field mappings and data types for the data warehouse.']}], 'duration': 438.896, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM83665786.jpg', 'highlights': ['Data extraction to identify customers purchasing less than 10 items per order for targeted marketing.', 'Using customer ID to obtain customer details for personalized marketing strategies.', 'Introduction of talent interface for data integration and job design using GUI and Oracle SQL database.', 'Demonstration of importing data from databases using drag-and-drop functionality on the UI.', 'Configuration details for importing data from the Oracle database, including host, port number, and username.', 'Insights into options and configuration details for importing data from different sources and mapping fields.']}, {'end': 5923.774, 'segs': [{'end': 4298.944, 'src': 'embed', 'start': 4271.305, 'weight': 2, 'content': [{'end': 4275.088, 'text': "let's have the description also and say finish.", 'start': 4271.305, 'duration': 3.783}, {'end': 4277.089, 'text': 'okay, we gotta choose a database type over here.', 'start': 4275.088, 'duration': 2.001}, {'end': 4281.832, 'text': 'okay, now, the database type that I want to import is that of Oracle.', 'start': 4277.089, 'duration': 4.743}, {'end': 4285.995, 'text': "right, and I'm gonna do it with the help of Oracle with the service name.", 'start': 4281.832, 'duration': 4.163}, {'end': 4291.397, 'text': 'So when you choose that, you have your login, you have your password and all these details.', 'start': 4287.554, 'duration': 3.843}, {'end': 4293.86, 'text': 'So in place of your login, give your username here.', 'start': 4291.417, 'duration': 2.443}, {'end': 4295.741, 'text': "So in my case, it's Vardhan.", 'start': 4294.56, 'duration': 1.181}, {'end': 4298.944, 'text': 'Okay, password is this one.', 'start': 4296.522, 'duration': 2.422}], 'summary': 'Select oracle as the database type and use username vardhan for login.', 'duration': 27.639, 'max_score': 4271.305, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM84271305.jpg'}, {'end': 4391.987, 'src': 'embed', 'start': 4353.741, 'weight': 0, 'content': [{'end': 4360.368, 'text': 'Now as you can see, my DB input here, this is a metadata which has these properties already inbuilt.', 'start': 4353.741, 'duration': 6.627}, {'end': 4367.059, 'text': "So in case of this one, I had to give it manually, but here in my metadata, I specified it once and it's already replicated.", 'start': 4361.195, 'duration': 5.864}, {'end': 4372.342, 'text': 'So the advantage here is that even if there are 10 different tables from which you want to get data in,', 'start': 4367.359, 'duration': 4.983}, {'end': 4375.424, 'text': "then you don't need to add 10 different database connections in your talent.", 'start': 4372.342, 'duration': 3.082}, {'end': 4378.006, 'text': "When you're creating your metadata.", 'start': 4376.065, 'duration': 1.941}, {'end': 4384.184, 'text': 'if you create the database connection once And then using that one time, you can establish 10 different connections.', 'start': 4378.006, 'duration': 6.178}, {'end': 4388.245, 'text': 'You can get it down from 10 different tables or replicate the same thing again and again.', 'start': 4384.364, 'duration': 3.881}, {'end': 4391.987, 'text': "So that is the advantage here, right? So that's the thing.", 'start': 4388.605, 'duration': 3.382}], 'summary': 'Using metadata for database connections reduces redundancy and allows for establishing multiple connections with just one setup.', 'duration': 38.246, 'max_score': 4353.741, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM84353741.jpg'}, {'end': 4455.534, 'src': 'embed', 'start': 4424.277, 'weight': 3, 'content': [{'end': 4429.099, 'text': "Now going back to my talent, now that I've set this, let me go and edit my schema now.", 'start': 4424.277, 'duration': 4.822}, {'end': 4432.821, 'text': "So where do I have my schema option? Okay, it's right here.", 'start': 4430.6, 'duration': 2.221}, {'end': 4437.104, 'text': "So I'm gonna say edit schema, and I'm gonna configure all the different columns and fields over here.", 'start': 4433.002, 'duration': 4.102}, {'end': 4441.128, 'text': "If I'm not wrong, there are five different columns here.", 'start': 4437.907, 'duration': 3.221}, {'end': 4445.81, 'text': 'Invoice number, description, quantity, customer ID, and product.', 'start': 4441.508, 'duration': 4.302}, {'end': 4447.811, 'text': "I'm gonna rename everything here.", 'start': 4446.41, 'duration': 1.401}, {'end': 4455.534, 'text': "I'm gonna say invoice number, description, quantity.", 'start': 4448.011, 'duration': 7.523}], 'summary': 'Editing schema with five columns: invoice number, description, quantity, customer id, and product.', 'duration': 31.257, 'max_score': 4424.277, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM84424277.jpg'}, {'end': 4882.696, 'src': 'embed', 'start': 4853.877, 'weight': 4, 'content': [{'end': 4856.74, 'text': "As per my problem statement, there's no need for a description.", 'start': 4853.877, 'duration': 2.863}, {'end': 4862.346, 'text': 'Similarly, even product ID is not really needed for me, so I can actually remove this also.', 'start': 4857.621, 'duration': 4.725}, {'end': 4865.61, 'text': 'So I have these, so these will basically be my output fields.', 'start': 4862.787, 'duration': 2.823}, {'end': 4870.435, 'text': 'My customer ID, customer name, customer phone, customer email address, invoice number, and quantity.', 'start': 4866.03, 'duration': 4.405}, {'end': 4875.343, 'text': 'Right. so these will be my output fields and another up set my schema.', 'start': 4870.835, 'duration': 4.508}, {'end': 4877.207, 'text': 'even the length has been auto picked right.', 'start': 4875.343, 'duration': 1.864}, {'end': 4880.172, 'text': 'so This is the nice thing about talent.', 'start': 4877.207, 'duration': 2.965}, {'end': 4882.696, 'text': "things are pretty simple and it's all straightforward, so you can just click on.", 'start': 4880.172, 'duration': 2.524}], 'summary': 'Output fields include customer id, name, phone, email, invoice number, and quantity.', 'duration': 28.819, 'max_score': 4853.877, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM84853877.jpg'}, {'end': 5025.203, 'src': 'embed', 'start': 4994.887, 'weight': 5, 'content': [{'end': 4997.068, 'text': 'The data that comes in can be of any data type.', 'start': 4994.887, 'duration': 2.181}, {'end': 5004.012, 'text': 'But the data that I store or the data that I finally export, that I can change the data type of that particular data.', 'start': 4997.869, 'duration': 6.143}, {'end': 5005.333, 'text': "So that's the advantage.", 'start': 5004.273, 'duration': 1.06}, {'end': 5012.317, 'text': "So quantity comes in the form of varchar2 data type and I'm converting it into integer from inside talent.", 'start': 5005.353, 'duration': 6.964}, {'end': 5015.259, 'text': "So that's the thing about advantage with talent.", 'start': 5012.898, 'duration': 2.361}, {'end': 5018.381, 'text': "So I'm just gonna say okay now.", 'start': 5015.559, 'duration': 2.822}, {'end': 5025.203, 'text': "So, now that I've done this, I think I'm done everything with respect to T join.", 'start': 5018.501, 'duration': 6.702}], 'summary': 'Data types can be modified in talend for better integration, such as converting varchar2 to integer.', 'duration': 30.316, 'max_score': 4994.887, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM84994887.jpg'}, {'end': 5109.709, 'src': 'embed', 'start': 5080.729, 'weight': 6, 'content': [{'end': 5087.338, 'text': "So let's say we want to find those people who have less than 10 transactions or whose quantity is less than 10.", 'start': 5080.729, 'duration': 6.609}, {'end': 5089.419, 'text': 'right who purchase less than 10 items.', 'start': 5087.338, 'duration': 2.081}, {'end': 5100.824, 'text': 'so for that we can just go back to our database and create another filter or component here which would filter out those people who have less than who have purchased less than 10 items per transaction.', 'start': 5089.419, 'duration': 11.405}, {'end': 5109.709, 'text': "right. so for that let's again go back to my uh palette here and search for this component filter.", 'start': 5100.824, 'duration': 8.885}], 'summary': 'Identifying individuals with less than 10 transactions or purchases.', 'duration': 28.98, 'max_score': 5080.729, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM85080729.jpg'}, {'end': 5348.06, 'src': 'embed', 'start': 5314.199, 'weight': 8, 'content': [{'end': 5317.34, 'text': "Okay, when I executed that, it's showing me an error over here.", 'start': 5314.199, 'duration': 3.141}, {'end': 5329.053, 'text': 'Okay, so the error it shows us that the error in the competence properties type mismatch cannot convert from string to integer Okay, not a big deal.', 'start': 5320.427, 'duration': 8.626}, {'end': 5333.455, 'text': 'I can fix this error.', 'start': 5329.453, 'duration': 4.002}, {'end': 5338.819, 'text': 'Okay, we can fix that by first of all checking our schema right over here.', 'start': 5333.455, 'duration': 5.364}, {'end': 5348.06, 'text': 'if you see the schema here, it says Quantity is type integer right, but over here Quantity is still type string.', 'start': 5338.819, 'duration': 9.241}], 'summary': 'Error: competence properties type mismatch - quantity type is string, should be integer.', 'duration': 33.861, 'max_score': 5314.199, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM85314199.jpg'}, {'end': 5643.452, 'src': 'embed', 'start': 5611.203, 'weight': 9, 'content': [{'end': 5612.984, 'text': "I'm gonna say define all the columns size.", 'start': 5611.203, 'duration': 1.781}, {'end': 5618.827, 'text': "I'm gonna say include header, and yep, I'm gonna save this and run it again.", 'start': 5613.124, 'duration': 5.703}, {'end': 5622.909, 'text': "When I run it again, I'll have the same list of fields over here.", 'start': 5618.847, 'duration': 4.062}, {'end': 5625.33, 'text': "I'll have the same rows over here.", 'start': 5623.609, 'duration': 1.721}, {'end': 5630.612, 'text': 'So as you can see, the same 841 rows have been added to this file.', 'start': 5626.69, 'duration': 3.922}, {'end': 5632.113, 'text': 'The filtered results are here.', 'start': 5630.632, 'duration': 1.481}, {'end': 5637.769, 'text': 'and the remaining ones which were rejected, they have gone to my new file, and this file is called the rejected one.', 'start': 5632.566, 'duration': 5.203}, {'end': 5643.452, 'text': 'So if I go back to that same path, this was the original one, and the rejected data is here.', 'start': 5638.169, 'duration': 5.283}], 'summary': '841 rows added, data filtered and rejected, saved separately.', 'duration': 32.249, 'max_score': 5611.203, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM85611203.jpg'}, {'end': 5766.655, 'src': 'embed', 'start': 5740.8, 'weight': 10, 'content': [{'end': 5750.781, 'text': "And the table name, let's say rejected output, okay? Well, I'm guessing there's nothing by that name.", 'start': 5740.8, 'duration': 9.981}, {'end': 5753.383, 'text': 'Yep, there is nothing here by that name.', 'start': 5751.302, 'duration': 2.081}, {'end': 5759.469, 'text': 'So we have customers, final table, and transactions, right? So we will have a new table that will be formed over there.', 'start': 5753.584, 'duration': 5.885}, {'end': 5763.012, 'text': 'So the action is gonna be insert.', 'start': 5760.61, 'duration': 2.402}, {'end': 5766.655, 'text': 'Supposing you wanna insert or update an existing file, then you can choose the various options,', 'start': 5763.032, 'duration': 3.623}], 'summary': 'Creating a new table named rejected output for customers and transactions with insert action.', 'duration': 25.855, 'max_score': 5740.8, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM85740800.jpg'}, {'end': 5885.183, 'src': 'embed', 'start': 5857.097, 'weight': 7, 'content': [{'end': 5862.168, 'text': 'We have the ETL, we have data marts, we have the metadata, and all these things.', 'start': 5857.097, 'duration': 5.071}, {'end': 5863.329, 'text': 'And then, finally,', 'start': 5862.688, 'duration': 0.641}, {'end': 5872.498, 'text': 'we spoke about the architecture of the data warehouse and we finished off the session with the hands-on demonstration of how to populate your data warehouse using talent.', 'start': 5863.329, 'duration': 9.169}, {'end': 5876.86, 'text': "Guys, that's it, and thank you for being in the session.", 'start': 5873.959, 'duration': 2.901}, {'end': 5878.741, 'text': 'That brings us to the end of the session today.', 'start': 5877.4, 'duration': 1.341}, {'end': 5885.183, 'text': "Probably we'll have another session on data warehousing, and I'll talk about more advanced concepts like schemas.", 'start': 5879.161, 'duration': 6.022}], 'summary': 'Covered etl, data marts, metadata, and data warehouse architecture, followed by a hands-on demonstration using talend. next session may cover advanced concepts like schemas.', 'duration': 28.086, 'max_score': 5857.097, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM85857097.jpg'}], 'start': 4104.863, 'title': 'Talend data management', 'summary': 'Covers setting up oracle database connection, table and schema setup, data schema editing and filtering, and data warehousing using talend, with examples such as configuring schema with five columns, filtering data with tfilterrow, and storing 841 rows in a database.', 'chapters': [{'end': 4391.987, 'start': 4104.863, 'title': 'Setting up oracle database connection', 'summary': 'Explains the process of setting up a connection to an oracle database in talend, demonstrating how to set the default query, run a query in oracle, configure inputs for different tables, and use metadata to create a standard input for data coming in from a particular database.', 'duration': 287.124, 'highlights': ['The advantage of using metadata is that you can create a standard input for data from a particular database, which can be used in any number of jobs in the project, eliminating the need to add multiple database connections in Talend. The advantage of using metadata is that you can create a standard input for data from a particular database, which can be used in any number of jobs in the project, eliminating the need to add multiple database connections in Talend.', 'By using metadata, the properties of the connection are specified once and automatically replicated, saving time and effort when establishing multiple connections to different tables. By using metadata, the properties of the connection are specified once and automatically replicated, saving time and effort when establishing multiple connections to different tables.', 'The process of setting up the Oracle database connection involves configuring inputs for different tables, specifying the database type, login details, and verifying the connection to successfully communicate with Oracle. The process of setting up the Oracle database connection involves configuring inputs for different tables, specifying the database type, login details, and verifying the connection to successfully communicate with Oracle.']}, {'end': 4816.708, 'start': 4392.387, 'title': 'Setting up table and schema in talend', 'summary': 'Covers the process of setting up a transactions table in talend, configuring the schema with five different columns, verifying data import, and merging tables using t join component.', 'duration': 424.321, 'highlights': ['Configuring schema with five different columns: invoice number, description, quantity, customer ID, and product ID. The speaker mentions the configuration of the schema with five different columns, namely invoice number, description, quantity, customer ID, and product ID.', 'Verifying data import and mapping details from the source to TLOG row. The process of verifying data import is described, along with the mapping of details from the source to the TLOG row.', 'Demonstrating the use of T join component to merge tables and configuring schema for the join. The use of the T join component to merge tables and the configuration of the schema for the join is explained.']}, {'end': 5260.687, 'start': 4817.309, 'title': 'Data schema editing and filtering', 'summary': 'Covers the process of editing the data schema, including removing unnecessary fields and setting the output fields, and filtering data based on the quantity to find customers with less than 10 transactions, using tjoin, tfilterrow, and tfileoutputexcel components, with a dataset of approximately 50,000 rows from input one and 10,000 rows from input two.', 'duration': 443.378, 'highlights': ['The process of editing the data schema involves removing unnecessary fields such as description and product ID, setting the output fields to customer ID, customer name, customer phone, customer email address, invoice number, and quantity, and auto picking the length using talent. Unnecessary fields like description and product ID are removed from the data schema, and the output fields are set to specific attributes, streamlining the data for further processing.', 'The ability to change the data type of the quantity field from varchar2 to integer within talent, allowing for data conversion without altering the original data. The talent platform enables changing the data type of the quantity field from varchar2 to integer, facilitating data manipulation without affecting the original dataset.', 'Filtering the data based on the quantity to find customers with less than 10 transactions using the TFilterRow component, and setting up TFileOutputExcel to store the filtered results. The TFilterRow component is used to filter data based on the quantity to identify customers with less than 10 transactions, and the TFileOutputExcel is set up to store the filtered results for further analysis and processing.']}, {'end': 5923.774, 'start': 5261.027, 'title': 'Data warehousing with talend', 'summary': 'Demonstrated the use of talend to filter and store data into an excel file and a database, with errors encountered and resolved, resulting in 8342 rows filtered, 841 rows stored, and a new table created.', 'duration': 662.747, 'highlights': ['The chapter demonstrated the use of Talend to filter and store data into an Excel file and a database. The demonstration showcased the practical application of Talend in filtering and storing data into an Excel file and a database.', 'Errors were encountered and resolved during the process. Various errors such as type mismatch and conversion issues were identified and resolved during the process.', '8342 rows were filtered and 841 rows were stored in the Excel file. A total of 8342 rows were filtered, with 841 rows successfully stored in the Excel file.', "A new table was created in the database to store the rejected data. A new table named 'rejected output' was created in the database to store the rejected data, with the action being an insert."]}], 'duration': 1818.911, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/J326LIUrZM8/pics/J326LIUrZM84104863.jpg', 'highlights': ['The advantage of using metadata is that you can create a standard input for data from a particular database, which can be used in any number of jobs in the project, eliminating the need to add multiple database connections in Talend.', 'By using metadata, the properties of the connection are specified once and automatically replicated, saving time and effort when establishing multiple connections to different tables.', 'The process of setting up the Oracle database connection involves configuring inputs for different tables, specifying the database type, login details, and verifying the connection to successfully communicate with Oracle.', 'Configuring schema with five different columns: invoice number, description, quantity, customer ID, and product ID.', 'The process of editing the data schema involves removing unnecessary fields such as description and product ID, setting the output fields to customer ID, customer name, customer phone, customer email address, invoice number, and quantity, and auto picking the length using talent.', 'The ability to change the data type of the quantity field from varchar2 to integer within talent, allowing for data conversion without altering the original data.', 'Filtering the data based on the quantity to find customers with less than 10 transactions using the TFilterRow component, and setting up TFileOutputExcel to store the filtered results.', 'The chapter demonstrated the use of Talend to filter and store data into an Excel file and a database.', 'Errors were encountered and resolved during the process.', '8342 rows were filtered and 841 rows were stored in the Excel file.', 'A new table was created in the database to store the rejected data.']}], 'highlights': ['Data warehousing is a vital subset of business intelligence, interlinked with OLTP and OLAP.', 'ETL strategy converts data from a database to a data warehouse, with detailed explanations.', 'Successful companies emphasize the importance of effective planning and data-driven strategies.', 'Data warehousing transforms raw data into useful information for business analysis.', 'Data warehouse technology plays a key role in extracting, transforming, and integrating data from operational systems.', 'Data warehousing provides readable information, making it easier to understand and use.', 'Data browsing in a data warehouse is faster and more accurate compared to a regular database.', 'Data integration involves standardizing and removing inconsistencies from data from multiple operational systems.', 'Data warehouse stores data by business subject, enabling integrated and single-view access to data.', 'Metadata defines source data, automates data import processes, and facilitates business logic transformation.', 'ETL process involves extracting, transforming, and loading data into a target data warehouse.', 'Data warehouse contains raw data, metadata, and aggregate data, enabling OLAP for analysis.', 'Data warehouse can be divided into data marts for different user groups to access specific data.', 'Data warehouse demonstration using Talend BI illustrates importing data from an Oracle database.', 'Primary and foreign key concept for linking tables and ensuring data integrity are explained.', 'Data extraction to identify customers purchasing less than 10 items per order for targeted marketing.', 'Using customer ID to obtain customer details for personalized marketing strategies.', 'Introduction of Talend interface for data integration and job design using GUI and Oracle SQL database.', 'Advantage of using metadata is creating a standard input for data from a particular database.', 'Configuring schema with five different columns: invoice number, description, quantity, customer ID, and product ID.', 'Ability to change the data type of the quantity field from varchar2 to integer within Talend.', 'Filtering the data based on the quantity to find customers with less than 10 transactions using the TFilterRow component.', 'Errors encountered and resolved during the process.', '8342 rows were filtered and 841 rows were stored in the Excel file.', 'A new table was created in the database to store the rejected data.']}