title
8.5: Saving Data to JSON File with Node.js - Programming with Text
description
In this video, I discuss how to work with a very simple database with in the API example using local JSON files and the node.js "fs"(file-system) package.
Next video: https://youtu.be/4zr8j-jeU_M
http://shiffman.net/a2z/node-api/
To know more on this subject, visit my material on Server-side Programming and Node: http://shiffman.net/a2z/server-node/
Course url: http://shiffman.net/a2z/
Support this channel on Patreon: https://patreon.com/codingtrain
Send me your questions and coding challenges!: https://github.com/CodingTrain/Rainbow-Topics
Contact: https://twitter.com/shiffman
GitHub Repo with all the info for Programming from A to Z: https://github.com/shiffman/A2Z-F16
Links discussed in this session:
Node.js: https://nodejs.org/
Node.js "fs" Documentation: https://nodejs.org/api/fs.html
Express.js: http://expressjs.com/
Source Code for the all Video Lessons: https://github.com/CodingTrain/Rainbow-Code
p5.js: https://p5js.org/
Processing: https://processing.org
For More Programming from A to Z videos: https://www.youtube.com/user/shiffman/playlists?shelf_id=11&view=50&sort=dd
For More Coding Challenges: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
Help us caption & translate this video!
http://amara.org/v/0WvN/
📄 Code of Conduct: https://github.com/CodingTrain/Code-of-Conduct
detail
{'title': '8.5: Saving Data to JSON File with Node.js - Programming with Text', 'heatmap': [{'end': 387.968, 'start': 376.873, 'weight': 1}, {'end': 453.033, 'start': 427.614, 'weight': 0.794}, {'end': 732.316, 'start': 683.376, 'weight': 0.714}, {'end': 829.625, 'start': 794.526, 'weight': 0.711}], 'summary': 'Covers adding persistence functionality to a node.js api, exploring the use of text and json files for data storage, node.js integration with firebase, couchdb, mongodb, and nedb, and file operations in a node.js environment, including synchronous vs asynchronous operations, data read/write, and basic api data saving concepts.', 'chapters': [{'end': 167.564, 'segs': [{'end': 79.866, 'src': 'embed', 'start': 49.823, 'weight': 0, 'content': [{'end': 55.248, 'text': 'So I need some mechanism by which I can save the data forever.', 'start': 49.823, 'duration': 5.425}, {'end': 59.495, 'text': "whether or not I'm running the server, quitting the server, not just in memory.", 'start': 56.593, 'duration': 2.902}, {'end': 62.056, 'text': 'And the way that this is done is typically with a database.', 'start': 59.775, 'duration': 2.281}, {'end': 64.757, 'text': 'Now, a database is a big topic.', 'start': 62.116, 'duration': 2.641}, {'end': 72.722, 'text': "And I expect that in the, what I imagine, future amount of time that I have in my life to make videos, I'll get into a lot of different facets of it.", 'start': 64.837, 'duration': 7.885}, {'end': 79.866, 'text': 'So what do we have so far is we have just data in memory.', 'start': 73.402, 'duration': 6.464}], 'summary': 'Data needs to be saved permanently using a database to ensure persistence beyond server runtime.', 'duration': 30.043, 'max_score': 49.823, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY49823.jpg'}, {'end': 139.948, 'src': 'embed', 'start': 105.877, 'weight': 1, 'content': [{'end': 109.498, 'text': 'And I could just save all the stuff and load that when I run the program.', 'start': 105.877, 'duration': 3.621}, {'end': 115.211, 'text': 'A way to make this even easier is to actually save this to a JSON file.', 'start': 110.928, 'duration': 4.283}, {'end': 118.253, 'text': 'JSON standing for JavaScript Object Notation.', 'start': 115.531, 'duration': 2.722}, {'end': 125.618, 'text': 'So this data, if you look at this, this is JavaScript Object Notation.', 'start': 118.753, 'duration': 6.865}, {'end': 130.561, 'text': "This syntax of having a variable full of key value pairs, that's JSON.", 'start': 126.018, 'duration': 4.543}, {'end': 139.948, 'text': 'So I can actually, nice option is just save the data to a JSON file and then load that JSON file every time the server starts.', 'start': 130.862, 'duration': 9.086}], 'summary': 'Saving data to a json file for easy loading and accessing when running the program.', 'duration': 34.071, 'max_score': 105.877, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY105877.jpg'}, {'end': 180.921, 'src': 'embed', 'start': 150.153, 'weight': 2, 'content': [{'end': 153.794, 'text': 'I have to load this giant text file and save this giant text file all the time.', 'start': 150.153, 'duration': 3.641}, {'end': 155.175, 'text': "That's not going to work very well.", 'start': 154.054, 'duration': 1.121}, {'end': 161.179, 'text': 'if I care about security and I have private data, just having it all just sitting there in a big text file, like text file,', 'start': 155.875, 'duration': 5.304}, {'end': 163.08, 'text': "full of everybody's logins and passwords.", 'start': 161.179, 'duration': 1.901}, {'end': 164.141, 'text': "that's not going to work very well.", 'start': 163.08, 'duration': 1.061}, {'end': 167.564, 'text': "So there are a lot of reasons why this isn't a particular great solution.", 'start': 164.381, 'duration': 3.183}, {'end': 174.509, 'text': 'But for a quick and dirty project, for understanding how things work, playing around in Node, I think this is going to be a great demonstration.', 'start': 168.064, 'duration': 6.445}, {'end': 180.921, 'text': 'But I will be making videos in the future that look at other database systems, namely one called Firebase.', 'start': 175.009, 'duration': 5.912}], 'summary': 'Loading and saving giant text files is not secure. firebase will be explored in future videos.', 'duration': 30.768, 'max_score': 150.153, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY150153.jpg'}], 'start': 0.229, 'title': 'Adding persistence functionality to a node.js api', 'summary': 'Explores the use of text and json files for data storage in a node.js api, highlighting limitations and security concerns associated with this approach.', 'chapters': [{'end': 167.564, 'start': 0.229, 'title': 'Building api with persistence', 'summary': 'Discusses the importance of adding persistence functionality to a node.js api, exploring the use of text and json files for data storage and the limitations and security concerns associated with this approach.', 'duration': 167.335, 'highlights': ['The significance of adding persistence functionality to the Node.js API is emphasized as the current data is lost once the server is quit, necessitating the need for data to be saved permanently. Importance of persistence for permanent data storage', 'The use of text and JSON files for data storage is explained, with JSON files being identified as a more convenient option due to their JavaScript Object Notation structure. Explanation of using text and JSON files, preference for JSON due to its structure', 'Limitations of using text or JSON files for storage are highlighted, particularly in handling massive data sets and ensuring security for private data. Limitations of text and JSON files in handling large data sets, security concerns']}], 'duration': 167.335, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY229.jpg', 'highlights': ['Importance of persistence for permanent data storage', 'Explanation of using text and JSON files, preference for JSON due to its structure', 'Limitations of text and JSON files in handling large data sets, security concerns']}, {'end': 478.047, 'segs': [{'end': 192.137, 'src': 'embed', 'start': 168.064, 'weight': 0, 'content': [{'end': 174.509, 'text': 'But for a quick and dirty project, for understanding how things work, playing around in Node, I think this is going to be a great demonstration.', 'start': 168.064, 'duration': 6.445}, {'end': 180.921, 'text': 'But I will be making videos in the future that look at other database systems, namely one called Firebase.', 'start': 175.009, 'duration': 5.912}, {'end': 189.855, 'text': "Firebase is something that's referred to, it's a Google product, database as service, meaning Firebase, you don't have to have your own server.", 'start': 181.402, 'duration': 8.453}, {'end': 192.137, 'text': "You're just a program, and you're keeping track of stuff.", 'start': 189.895, 'duration': 2.242}], 'summary': 'Node is great for quick projects, firebase is a google product for database as a service.', 'duration': 24.073, 'max_score': 168.064, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY168064.jpg'}, {'end': 241.327, 'src': 'embed', 'start': 206.328, 'weight': 1, 'content': [{'end': 210.572, 'text': 'And then, of course, you could use a, quote unquote, a real database or some type of database system.', 'start': 206.328, 'duration': 4.244}, {'end': 213.194, 'text': 'There are other databases that service products, by the way.', 'start': 210.592, 'duration': 2.602}, {'end': 214.014, 'text': 'So you can find those.', 'start': 213.234, 'duration': 0.78}, {'end': 217.057, 'text': 'But you could use something like CouchDB.', 'start': 214.335, 'duration': 2.722}, {'end': 225.762, 'text': 'or MongoDB, or another database that I actually like, which is very simple, which is called Nedb, I think.', 'start': 218.2, 'duration': 7.562}, {'end': 231.184, 'text': 'So these are all database systems that you can use with Node or other server-side programming frameworks.', 'start': 226.103, 'duration': 5.081}, {'end': 237.226, 'text': 'So at some point, somebody remind me, hey, definitely I have a whole bunch of examples already for Firebase, so I intend to do that.', 'start': 231.224, 'duration': 6.002}, {'end': 241.327, 'text': "But I'd love to look into this kind of stuff and make some examples with that as well.", 'start': 237.466, 'duration': 3.861}], 'summary': 'Various database options like couchdb, mongodb, and nedb can be used with node or other server-side programming frameworks.', 'duration': 34.999, 'max_score': 206.328, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY206328.jpg'}, {'end': 325.733, 'src': 'embed', 'start': 299.296, 'weight': 3, 'content': [{'end': 306.943, 'text': 'I want to do something like if I were in client-side P5 LAN, I would just say load JSON words.json.', 'start': 299.296, 'duration': 7.647}, {'end': 309.165, 'text': "I want just to load what's ever in that file and stick it in words.", 'start': 306.963, 'duration': 2.202}, {'end': 310.566, 'text': 'But this is not node code.', 'start': 309.205, 'duration': 1.361}, {'end': 312.788, 'text': 'This is P5 code.', 'start': 311.026, 'duration': 1.762}, {'end': 314.73, 'text': 'So I need a different set of syntax for that.', 'start': 312.808, 'duration': 1.922}, {'end': 319.671, 'text': 'So how do I do that in Node? Well, what I need to use is the FS module.', 'start': 315.31, 'duration': 4.361}, {'end': 323.673, 'text': 'So FS, Node, Node.', 'start': 319.731, 'duration': 3.942}, {'end': 325.733, 'text': "I'm just going to Google FS and Node.", 'start': 323.793, 'duration': 1.94}], 'summary': 'Using fs module in node to load json file into variable', 'duration': 26.437, 'max_score': 299.296, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY299296.jpg'}, {'end': 408.184, 'src': 'heatmap', 'start': 376.873, 'weight': 1, 'content': [{'end': 380.079, 'text': "And then I'm going to say console.log words.", 'start': 376.873, 'duration': 3.206}, {'end': 382.624, 'text': "Let's see what's happening here.", 'start': 381.462, 'duration': 1.162}, {'end': 384.307, 'text': 'Ah, OK.', 'start': 384.006, 'duration': 0.301}, {'end': 387.247, 'text': 'So it runs, and look, I read the file.', 'start': 385.526, 'duration': 1.721}, {'end': 387.968, 'text': "There's the file.", 'start': 387.347, 'duration': 0.621}, {'end': 389.329, 'text': "Perfect There's my contents.", 'start': 388.048, 'duration': 1.281}, {'end': 397.275, 'text': 'What is that? So one thing you have to realize is the node file system package is just reading the raw data files and writing the raw data out.', 'start': 389.489, 'duration': 7.786}, {'end': 400.398, 'text': "It doesn't know, oh, I want this to be like JSON and all of that.", 'start': 397.315, 'duration': 3.083}, {'end': 408.184, 'text': 'So if I have that raw data from the file, but I actually want it to be JavaScript and JavaScript object, I need to parse it.', 'start': 400.678, 'duration': 7.506}], 'summary': 'Node file system package reads and writes raw data, necessitating parsing for javascript objects.', 'duration': 31.311, 'max_score': 376.873, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY376873.jpg'}, {'end': 414.828, 'src': 'embed', 'start': 389.489, 'weight': 4, 'content': [{'end': 397.275, 'text': 'What is that? So one thing you have to realize is the node file system package is just reading the raw data files and writing the raw data out.', 'start': 389.489, 'duration': 7.786}, {'end': 400.398, 'text': "It doesn't know, oh, I want this to be like JSON and all of that.", 'start': 397.315, 'duration': 3.083}, {'end': 408.184, 'text': 'So if I have that raw data from the file, but I actually want it to be JavaScript and JavaScript object, I need to parse it.', 'start': 400.678, 'duration': 7.506}, {'end': 409.866, 'text': "And there's a quick and easy way of doing that.", 'start': 408.224, 'duration': 1.642}, {'end': 414.828, 'text': "So actually what I want to do is I'm going to say, I'm just going to say var data equals that.", 'start': 410.666, 'duration': 4.162}], 'summary': 'Node file system package reads and writes raw data, requiring parsing for javascript object conversion.', 'duration': 25.339, 'max_score': 389.489, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY389489.jpg'}, {'end': 458.421, 'src': 'heatmap', 'start': 427.614, 'weight': 0.794, 'content': [{'end': 430.595, 'text': 'When I have a JavaScript, I need to interpret it as a JavaScript object.', 'start': 427.614, 'duration': 2.981}, {'end': 431.776, 'text': 'before I can use it.', 'start': 431.035, 'duration': 0.741}, {'end': 437.861, 'text': 'When I have a JavaScript object and I want to save it to the file, I need to convert it to just plain old text and then save it to the file.', 'start': 432.116, 'duration': 5.745}, {'end': 440.002, 'text': "So let's look how that works now.", 'start': 438.221, 'duration': 1.781}, {'end': 441.123, 'text': 'And you can see, there we go.', 'start': 440.062, 'duration': 1.061}, {'end': 443.846, 'text': 'So now my server is reading that stuff.', 'start': 441.424, 'duration': 2.422}, {'end': 448.589, 'text': "So we've got step one in that everything should work as it did before.", 'start': 444.426, 'duration': 4.163}, {'end': 453.033, 'text': "All, and I could add something, but I'm still going to have the problem.", 'start': 450.451, 'duration': 2.582}, {'end': 458.421, 'text': "As soon as I quit the server, and relaunch it, anything that I've added will be gone.", 'start': 453.373, 'duration': 5.048}], 'summary': 'To save javascript object to file, convert it to text. server retains added data until relaunch.', 'duration': 30.807, 'max_score': 427.614, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY427614.jpg'}, {'end': 481.148, 'src': 'embed', 'start': 453.373, 'weight': 5, 'content': [{'end': 458.421, 'text': "As soon as I quit the server, and relaunch it, anything that I've added will be gone.", 'start': 453.373, 'duration': 5.048}, {'end': 460.781, 'text': 'So how do I now have persistence?', 'start': 458.741, 'duration': 2.04}, {'end': 466.403, 'text': 'Where in my code do I want to save data to the file itself?', 'start': 461.182, 'duration': 5.221}, {'end': 470.284, 'text': 'Well, I want to do that any time I add something new to this list.', 'start': 467.043, 'duration': 3.241}, {'end': 478.047, 'text': 'And if I go back to the server program, the only time I add something new to the list is right here under Add Word.', 'start': 470.645, 'duration': 7.402}, {'end': 481.148, 'text': 'So this is where I want to save data.', 'start': 478.307, 'duration': 2.841}], 'summary': 'To achieve persistence, save data to file after adding to list in server program.', 'duration': 27.775, 'max_score': 453.373, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY453373.jpg'}], 'start': 168.064, 'title': 'Node.js and firebase database systems', 'summary': 'Covers the use of node.js with firebase and other database systems like couchdb, mongodb, and nedb for server-side programming. it also explains the process of creating, reading, and parsing a json file in a node.js environment, including the use of the fs module and json parsing method.', 'chapters': [{'end': 241.327, 'start': 168.064, 'title': 'Node and firebase database systems', 'summary': 'Explains how to use node with different database systems, such as firebase, a google product that offers database as a service, and mentions other options like couchdb, mongodb, and nedb for server-side programming. the chapter also intends to provide examples with firebase in the future.', 'duration': 73.263, 'highlights': ['The chapter explains how to use Node with different database systems, such as Firebase, a Google product that offers database as a service. The chapter introduces Firebase, a database as a service product by Google, which allows easy data storage and retrieval without the need for a personal server.', 'Mentions other options like CouchDB, MongoDB, and Nedb for server-side programming. The chapter mentions other database systems like CouchDB, MongoDB, and Nedb that can be utilized for server-side programming.', 'The chapter also intends to provide examples with Firebase in the future. The chapter plans to offer examples using Firebase in the future, expanding on its usage in Node and server-side programming.']}, {'end': 478.047, 'start': 241.628, 'title': 'Node.js file system and json parsing', 'summary': 'Discusses the process of creating, reading, and parsing a json file in a node.js environment, covering the use of the fs module, the read file functions, and the json parsing method.', 'duration': 236.419, 'highlights': ['The process of creating and reading a JSON file in a Node.js environment is explained, emphasizing the use of the FS module and the read file functions. The chapter provides a detailed explanation of creating a JSON file in a Node.js environment and reading its contents using the FS module and the read file functions.', 'The importance of parsing raw data from a file into a JavaScript object using the JSON parse method is highlighted. The chapter emphasizes the significance of using the JSON parse method to convert raw data from a file into a JavaScript object for further utilization.', 'The need for persisting data by saving it to the file is discussed, particularly in the context of adding new content to a list. The chapter delves into the necessity of persisting data by saving it to the file, particularly when adding new content to a list in the server program.']}], 'duration': 309.983, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY168064.jpg', 'highlights': ['The chapter introduces Firebase, a database as a service product by Google, which allows easy data storage and retrieval without the need for a personal server.', 'The chapter mentions other database systems like CouchDB, MongoDB, and Nedb that can be utilized for server-side programming.', 'The chapter plans to offer examples using Firebase in the future, expanding on its usage in Node and server-side programming.', 'The chapter provides a detailed explanation of creating a JSON file in a Node.js environment and reading its contents using the FS module and the read file functions.', 'The chapter emphasizes the significance of using the JSON parse method to convert raw data from a file into a JavaScript object for further utilization.', 'The chapter delves into the necessity of persisting data by saving it to the file, particularly when adding new content to a list in the server program.']}, {'end': 1008.204, 'segs': [{'end': 556.427, 'src': 'embed', 'start': 532.984, 'weight': 0, 'content': [{'end': 539.768, 'text': "So I want to use the synchronous version, so I load the data, and I don't have to have a callback, and it makes writing the code a little bit simpler.", 'start': 532.984, 'duration': 6.784}, {'end': 548.1, 'text': "However, if I'm going to perform an action where I'm reading and writing to files while a user is making an API request,", 'start': 541.055, 'duration': 7.045}, {'end': 553.104, 'text': "I don't want to use the sync method, because that will actually lock up the server while it's waiting to do this operation.", 'start': 548.1, 'duration': 5.004}, {'end': 556.427, 'text': 'I want to use the non-sync asynchronous version,', 'start': 553.404, 'duration': 3.023}], 'summary': 'Prefer using non-sync asynchronous version for file operations to avoid server lock-up.', 'duration': 23.443, 'max_score': 532.984, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY532984.jpg'}, {'end': 735.117, 'src': 'heatmap', 'start': 673.489, 'weight': 1, 'content': [{'end': 676.751, 'text': "Thank you for your word, but let's look at, ah, crashed.", 'start': 673.489, 'duration': 3.262}, {'end': 680.354, 'text': 'Unexpected token, blah, blah, blah, object, object.', 'start': 677.292, 'duration': 3.062}, {'end': 683.036, 'text': "It couldn't figure out how to write that to a file right?", 'start': 680.434, 'duration': 2.602}, {'end': 688.28, 'text': "Because, once again, just as if I'm reading data from a file,", 'start': 683.376, 'duration': 4.904}, {'end': 692.783, 'text': "I'm getting the raw bytes and I need to parse it as a JavaScript object before I can use it.", 'start': 688.28, 'duration': 4.503}, {'end': 697.668, 'text': 'Now what I need to do is I need to turn it into text-based data before I write it to the file.', 'start': 693.343, 'duration': 4.325}, {'end': 705.056, 'text': 'And the way to do that I can say just var data equals json dot the opposite of parse or the inverse is stringify.', 'start': 698.168, 'duration': 6.888}, {'end': 711.73, 'text': 'So I can say stringify words and then write that data to the file.', 'start': 705.697, 'duration': 6.033}, {'end': 720.953, 'text': "So, now that I've done that now, one thing I want to do actually is I want to stop using nodemon, because nodemon restarts the server every time.", 'start': 712.07, 'duration': 8.883}, {'end': 724.174, 'text': 'oops, I have oops.', 'start': 720.953, 'duration': 3.221}, {'end': 732.316, 'text': "What's going on here? Error, undefined one, unexpected token, JSON.", 'start': 726.094, 'duration': 6.222}, {'end': 735.117, 'text': 'Oh, you know what? I messed up the file.', 'start': 733.236, 'duration': 1.881}], 'summary': 'Using json.stringify to write data to a file in javascript', 'duration': 61.628, 'max_score': 673.489, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY673489.jpg'}, {'end': 829.625, 'src': 'heatmap', 'start': 794.526, 'weight': 0.711, 'content': [{'end': 803.014, 'text': 'but I can use some other arguments and I can add, like I forget why you put null there to look at the documentation for stringify but two,', 'start': 794.526, 'duration': 8.488}, {'end': 806.117, 'text': 'meaning that I want to use two spaces for an indent.', 'start': 803.014, 'duration': 3.103}, {'end': 812.544, 'text': 'so if I do this and I restart the server and by the way, purple is already there,', 'start': 806.117, 'duration': 6.427}, {'end': 818.289, 'text': 'so purple is now there forever and I can go to pink and add the number six.', 'start': 812.544, 'duration': 5.745}, {'end': 820.508, 'text': 'Thank you for your word.', 'start': 819.346, 'duration': 1.162}, {'end': 823.875, 'text': 'And if I go back and I look at that, we can see there we go.', 'start': 820.849, 'duration': 3.026}, {'end': 829.625, 'text': 'So every time I go to that route, it rewrites the entire file with the current list of words.', 'start': 824.295, 'duration': 5.33}], 'summary': "Demonstrating adding 'purple' and 'pink' with numbers to a file, rewriting the entire file with the current list of words.", 'duration': 35.099, 'max_score': 794.526, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY794526.jpg'}, {'end': 859.092, 'src': 'embed', 'start': 830.891, 'weight': 6, 'content': [{'end': 835.115, 'text': 'Every time I quit the server and start the server back up, it reads the list of words.', 'start': 830.891, 'duration': 4.224}, {'end': 837.336, 'text': 'So this is the full round trip.', 'start': 835.335, 'duration': 2.001}, {'end': 841.8, 'text': "There's lots of inefficiencies and issues, again, with scalability and privacy.", 'start': 837.356, 'duration': 4.444}, {'end': 849.086, 'text': 'But this works for a simple project where you just want to save a high score list or a table of words in their sentiment score.', 'start': 842.08, 'duration': 7.006}, {'end': 850.427, 'text': 'You can do some of this.', 'start': 849.466, 'duration': 0.961}, {'end': 854.691, 'text': "And I'm sure you can imagine some other scenarios where just this basic idea is plenty good enough.", 'start': 850.447, 'duration': 4.244}, {'end': 859.092, 'text': 'Let me say another few things about this, though, before I move to the end of this video.', 'start': 855.091, 'duration': 4.001}], 'summary': 'Server process reads list of words, showing scalability issues. suitable for high score/sentiment score storage.', 'duration': 28.201, 'max_score': 830.891, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY830891.jpg'}, {'end': 895.024, 'src': 'embed', 'start': 871.134, 'weight': 4, 'content': [{'end': 878.256, 'text': 'So really, probably, I should wait to send a reply to the client who added this word until that file has finished being written.', 'start': 871.134, 'duration': 7.122}, {'end': 886.841, 'text': 'So I would say it probably makes sense to to put that in here.', 'start': 878.716, 'duration': 8.125}, {'end': 890.422, 'text': "And I'm going to send the reply actually.", 'start': 888.001, 'duration': 2.421}, {'end': 893.703, 'text': 'And because I had this error checking, I have to send the reply up here.', 'start': 890.582, 'duration': 3.121}, {'end': 895.024, 'text': "I could think about if there's a better way.", 'start': 893.723, 'duration': 1.301}], 'summary': 'Suggest delaying reply to client until file is written, considering better error checking.', 'duration': 23.89, 'max_score': 871.134, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY871134.jpg'}, {'end': 950.342, 'src': 'embed', 'start': 922.657, 'weight': 5, 'content': [{'end': 926.778, 'text': "So, in other words, sometimes, when you're making an API request or you're adding something,", 'start': 922.657, 'duration': 4.121}, {'end': 931.239, 'text': "you're sending some data to an API and you're doing that many, many times as a client.", 'start': 926.778, 'duration': 4.461}, {'end': 934.159, 'text': 'when you get a reply back, you need to match it with which one you sent.', 'start': 931.239, 'duration': 2.92}, {'end': 937.36, 'text': 'So if you get some information back that you can match, that can often be helpful.', 'start': 934.379, 'duration': 2.981}, {'end': 941.84, 'text': "So even though this is redundant and I don't personally need this information, I think it's useful to add.", 'start': 937.62, 'duration': 4.22}, {'end': 947.581, 'text': 'So I did sort of a very awkward thing where my variables all have the same name, but this will actually should work.', 'start': 942.24, 'duration': 5.341}, {'end': 950.342, 'text': "So let's restart the server one time.", 'start': 947.962, 'duration': 2.38}], 'summary': 'Frequent api requests require matching data for efficient processing. redundant but useful information added.', 'duration': 27.685, 'max_score': 922.657, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY922657.jpg'}, {'end': 1008.204, 'src': 'embed', 'start': 992.276, 'weight': 3, 'content': [{'end': 999.88, 'text': "So I've still got more to do in terms of building this API, which is actually get some text and produce a sentiment score.", 'start': 992.276, 'duration': 7.604}, {'end': 1000.6, 'text': 'I need to look at.', 'start': 999.9, 'duration': 0.7}, {'end': 1004.842, 'text': 'well, how would I build a client, a front end, that would actually interface and interact with the API?', 'start': 1000.6, 'duration': 4.242}, {'end': 1007.524, 'text': "But at least now we've seen a little bit about saving data.", 'start': 1005.142, 'duration': 2.382}, {'end': 1008.204, 'text': 'Thanks for watching.', 'start': 1007.684, 'duration': 0.52}], 'summary': 'Building api to analyze text sentiment, exploring client interface, and saving data.', 'duration': 15.928, 'max_score': 992.276, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY992276.jpg'}], 'start': 478.307, 'title': 'File operations and basic api data saving', 'summary': 'Explains synchronous vs asynchronous file operations, their impact on server performance, and using node.js for data read/write. it also discusses saving and retrieving data in a basic api, addressing scalability and privacy issues, and stressing error handling and data validation. these concepts are important for server performance and basic api development.', 'chapters': [{'end': 803.014, 'start': 478.307, 'title': 'Synchronous vs asynchronous file operations', 'summary': 'Explains the difference between synchronous and asynchronous file operations, highlighting the impact on server performance and illustrating the process of reading and writing data to files using node.js. it emphasizes the use of synchronous file operations for server startup and the need for error handling and data parsing for successful file write operations.', 'duration': 324.707, 'highlights': ['The importance of synchronous file operations for server startup and the need for error handling and data parsing for successful file write operations The chapter emphasizes the use of synchronous file operations when the server starts up to ensure that the next line of code does not execute until the data has been read. It also highlights the need for error handling and data parsing for successful file write operations.', 'The impact of using synchronous file operations on server performance It explains that using synchronous file operations can lock up the server while waiting to perform an operation, impacting server performance, and recommends using asynchronous file operations when reading and writing to files during user API requests.', 'Demonstration of writing data to a file using Node.js with error handling and data parsing The chapter demonstrates the process of writing data to a file using Node.js with error handling for scenarios where the given data is not complete and the need for data parsing to convert the JavaScript object into text-based data before writing it to the file.']}, {'end': 1008.204, 'start': 803.014, 'title': 'Saving data in a basic api', 'summary': 'Discusses the process of saving and retrieving data in a basic api, highlighting the inefficiencies and issues with scalability and privacy, but suitable for simple projects, and emphasizes the importance of error handling and data validation.', 'duration': 205.19, 'highlights': ['The chapter explains the process of saving and retrieving data in a basic API, suitable for simple projects like saving a high score list or a table of words with sentiment scores.', 'It emphasizes the importance of error handling and data validation, suggesting the need to wait to send a reply to the client until the file has finished being written.', 'The presenter acknowledges inefficiencies and issues with scalability and privacy in the discussed approach.', 'The chapter highlights the significance of an API sending back the data that has been sent to it, and the usefulness of receiving information back to match with the sent data in API requests.']}], 'duration': 529.897, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/6iZiqQZBQJY/pics/6iZiqQZBQJY478307.jpg', 'highlights': ['The importance of synchronous file operations for server startup and the need for error handling and data parsing for successful file write operations', 'Demonstration of writing data to a file using Node.js with error handling and data parsing', 'The impact of using synchronous file operations on server performance', 'The chapter explains the process of saving and retrieving data in a basic API, suitable for simple projects like saving a high score list or a table of words with sentiment scores', 'The chapter emphasizes the importance of error handling and data validation, suggesting the need to wait to send a reply to the client until the file has finished being written', 'The chapter highlights the significance of an API sending back the data that has been sent to it, and the usefulness of receiving information back to match with the sent data in API requests', 'The presenter acknowledges inefficiencies and issues with scalability and privacy in the discussed approach']}], 'highlights': ['The importance of synchronous file operations for server startup and the need for error handling and data parsing for successful file write operations', 'The chapter explains the process of saving and retrieving data in a basic API, suitable for simple projects like saving a high score list or a table of words with sentiment scores', 'The chapter emphasizes the importance of error handling and data validation, suggesting the need to wait to send a reply to the client until the file has finished being written', 'The chapter highlights the significance of an API sending back the data that has been sent to it, and the usefulness of receiving information back to match with the sent data in API requests', 'The presenter acknowledges inefficiencies and issues with scalability and privacy in the discussed approach', 'The chapter introduces Firebase, a database as a service product by Google, which allows easy data storage and retrieval without the need for a personal server.', 'The chapter mentions other database systems like CouchDB, MongoDB, and Nedb that can be utilized for server-side programming.', 'The chapter plans to offer examples using Firebase in the future, expanding on its usage in Node and server-side programming.', 'The chapter provides a detailed explanation of creating a JSON file in a Node.js environment and reading its contents using the FS module and the read file functions.', 'The chapter emphasizes the significance of using the JSON parse method to convert raw data from a file into a JavaScript object for further utilization.', 'The chapter delves into the necessity of persisting data by saving it to the file, particularly when adding new content to a list in the server program.', 'Importance of persistence for permanent data storage', 'Explanation of using text and JSON files, preference for JSON due to its structure', 'Limitations of text and JSON files in handling large data sets, security concerns']}