I was restoring a saved model in Tensorflow 1.5 when I was getting the error below.
DataLossError (see above for traceback): Unable to open table file …: Data loss: not an sstable (bad magic number):
Checkpoint v2 saves the checkpoint as 3 files (*.data-00000-of-00001, *.index, *.meta). So when restoring the checkpoint remove the extension and just use the filename *.ckpt
Sometimes finding the correct version of Python used in a virtual environment can be a problem. This blog post provides some good tips on how to fix the problem. http://takluyver.github.io/posts/i-cant-import-it.html
It often seems like technology company deals are so ugly that only the bankers could love them. That is why it was refreshing to see a deal that makes sense on a lot of levels. Reading a lot of the news reports I feel that there is a deeper narrative at play that has largely been glossed over. So here is a quick summary of the great options that this deal provides Amazon.
In the short-term this deal gives Amazon a power position over their competitors in food delivery. Instacart for example uses Whole Foods stores to get groceries and deliver them to customers. Assuming this deal continues, Amazon will benefit by getting a cut of the Instacart transactions. You can think of this as the Amazon tax. If Instacart starts making more money, Amazon can just raise prices via Whole Foods to get a larger cut of Instacart’s business. Instacart could also decide to move on to another grocer but Whole Foods has unique food selection. Plus, Amazon has a history of letting others user their infrastructure (e.g. AWS, fulfillment centers) and might prefer continually getting money from Instacart transactions rather than kicking them out. This deal also has short-term implications for Amazon’s current partnerships. Here in the greater Seattle area Amazon works with PCC which is another grocery store. Amazon can continue to work with PCC and use their ownership of Whole Foods as leverage in future negotiations. However, PCC will most likely be relegated to support burstable demand when the Whole Foods stores don’t have enough supply or are too busy. From a retail perspective, I wonder if Prime will enable Whole Foods to lose their “Whole Paycheck” reputation by using Prime as a driver of revenue in physical retail while lowering prices in the actual store. This is similar to Costco’s model.
Another interesting option is the stores being the fulfillment centers for groceries and perishable goods. Amazon has 99.4 million square feet of fulfillment centers (this also includes data centers which Amazon bundles in this number) in North America (source) and with Whole Foods they will now have an additional 17.8 million square feet spread across 456 stores in the USA, Canada and Europe (source). Whole Foods stores are known for being in prime locations in urban areas providing ideal locations for Amazon to reach customers in densely populated areas. These areas have higher costs and are not the type of places where a frugal retailer would open a fulfillment center. Closer locations to densely populated areas give Amazon the opportunity to provide other goods more easily to customers in 2 hours or less. Proximity is critical and the grocery store makes having a mini urban fulfillment center a profitable endeavor and not just a cost center. The more stuff you can get in this timeframe, the less likely you will be to go to the store, which leads to buying more stuff from Amazon and the flywheel will pick up speed. Amazon arrives at my door so often now I am starting recognize some of the drivers.
In the long run this is where we can see some of the more futuristic scenarios like Amazon Go. Amazon Go uses artificial intelligence so customers can grab what they need and go. As sexy as these scenarios are unless the technology is ready for prime time it is unlikely this was a strong driver for the deal. According to Amazon’s 8-K, they will be paying $13.7 billion which will be paid for in debt so any efficiency from AI won’t make this deal payoff right away.
Speaking of money, when you look at Amazon’s financials it makes sense that they would borrow the money. With $15.4 billion in cash at the end of the last quarter it highlights that Amazon is a technology company that has the financials of a retailer. Whole Foods free cash flow of $400 million last year is also not a spigot of cash that will make this deal a quick payoff either. Some argue that Amazon got Whole Foods for free because Amazon’s market capitalization increased by $11.2 billion today. This is a cash deal so the currency comparison in this case is apple and oranges unless they are going to have some secondary offerings. More importantly, Amazon has never been overly concerned about profits and this time should not be any different.
The reason why this deal makes sense in my opinion is really about the short, medium and long term optionality that sets them up for success regardless of how the food delivery industry turns out. It is for those reasons that I feel that Amazon truly made a strategic purchase in acquiring Whole Foods (pending regulatory approval of course).
Talk to you soon,
Orville | Twitter: @orville_m
One of my favorite things to do during the holidays is a coding project. Between work and family commitments it is possible to do side projects during the year but harder to just immerse yourself into a problem. My only criteria for the projects is that the technologies that I use have to be different than what I use at work. Normally I pick a technology that I am interested in and then find a project for it. Although I have learned a lot that way the approach I took this year was a lot better.
So here is a quick list of what I learned in how to do my annual holiday project:
- Be mission driven instead of technology focused. Normally I would think of a technology that I want to learn about and then find a problem to apply it to. This year there was a problem that has been on my mind that I wanted to address. Looking at a problem first you end up finding the best technology to address the problem which is often different than the original technology.
- Measure twice, cut once. If I was working on a project during crunch time I would use the best solution that I could think of and code away. However, with more time I could look at different alternatives before deciding. For example, I could look at different NLP papers for different approaches. Doing this seems like a waste of time when you don’t have a lot of it. Yet it resulted in me getting more done in less time because I could think about the trade-offs before I write the first line of code. That way I saved time from short-sighted algorithms or rewriting buggy code.
- Go back to school. Academic papers contain so many great insights that I will probably continue to read these to learn about new topics instead of just blogs. For this project, I spent a lot of time reading about word sense disambiguation, sentiment analysis and opinion mining. Funny how a simple question could lead to reading about so many different things. I have also been a long-time fan of MOOCs. Often I will listen to Udacity lectures to come up to speed on a topic and at times even do some of the assignments. That is how I became familiar with TensorFlow.
- Manage the clock. Maybe it is because I have been watching so much football (good luck next year UW) but I was always careful to manage my time throughout the project. I was firm that I needed to be code complete a few days before my vacation was over. My original thinking is that the project would go from something fun to a chore if I was trying to complete it while I was busy with work and other life commitments. The added benefit is that I became ruthless in prioritizing my time and even cut “cute” features to focus on what was most important.
- Focus on Minimal and Viable. When I talk to people building new products they always talk about the MVP. Once you start digging into details it is clear that they are preparing for a moon landing. Focus on the shortest path to viable and feel free to use lots of open source code.
- Not all open source projects are equal. Open source software has made it possible to create something amazing quickly. Before standing on the shoulder of giants you should make sure that the giant is in shape. For me that involved a few things including when was the last time someone worked on the project, why did the person create the project, has anyone contributed to it, etc. However, I found the most important check was to read the code itself. Was it readable, have error checking and do what it claims to. In a few instances, I leveraged repos that haven’t been touched in years but were of good quality.
This post could easily have been longer but these were a few things that popped out at me. At this point you might be wondering what did I create? Well I had so much fun with this one I will probably release it in the coming weeks and will create another blog post discussing the problem being solved and the solution I proposed. Now I need to finish getting ready for CES… everyday life continues. 🙂
Talk to you soon,
Orville | Twitter: @orville_m
While installing TensorFlow on a Mac running El Capitan I kept getting one of those crazy Python errors that are impossible to decipher. Thanks to Stackoverflow which always saves me while coding I found out the problem. It is apparently “due to the System Integrity Protection introduced in OS X El Capitan.” The fix is copied below for the next time I run into this error. Thank you for the tip Kof.
sudo pip install --upgrade $TF_BINARY_URL --user python
I am traveling home form Orlando where I presented at Microsoft’s Worldwide Partner Conference. It was great fun and I learned a lot from our partners. Since over 4,000 people from the Seattle area are here I had to change my travel routine. In this case that means traveling on American Airlines to get home.
American Airlines has an interesting checked baggage pricing structure that I listed below:
- One bag is $25. That is standard.
- Two bags are $60. Ok, this is reasonable to charge more for the second bag.
- Three bags are $210. Really going to charge that much for 3 bags.
Orlando is the home of Disney World so there are a lot of families meaning lots of bags. So of course a lot of people try to put huge bags in the overhead bin. This causes so many delays and frustration for the people waiting.
Hopefully American Airlines could adopt more reasonable price discrimination so they discourage a lot bags without creating problems for the travelers.
On the runway,
Often when using Solr, I forget what the command is to delete documents so I’m just keeping it here. This command will both delete the documents matched by the query and commit the deletion.
Query is format field:value. To delete all of your documents enter *:*
BTW – if you do this from a browser make sure that you close the browser window when you are finished. I once had Firefox crash and then when it came up I opted to recover the closed windows. This instantly executed this command and deleted all of my documents. it was not a good feeling.
Talk to you soon,
Orville | Twitter: @orville_m