Tech Book Face Off: Data Smart Vs. Python Machine Learning

After reading a few books on data science and a little bit about machine learning, I felt it was time to round out my studies in these subjects with a couple more books. I was hoping to get some more exposure to implementing different machine learning algorithms as well as diving deeper into how to effectively use the different Python tools for machine learning, and these two books seemed to fit the bill. The first book with the upside-down face, Data Smart: Using Data Science to Transform Data Into Insight by John W. Foreman, looked like it would fulfill the former goal and do it all in Excel, oddly enough. The second book with the right side-up face, Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow by Sebastian Raschka and Vahid Mirjalili, promised to address the second goal. Let's see how these two books complement each other and move the reader toward a better understanding of machine learning.

VS.

Data Smart

I must admit; I was somewhat hesitant to get this book. I was worried that presenting everything in Excel would be a bit too simple to really learn much about data science, but I needn't have been concerned. This book was an excellent read for multiple reasons, not least of which is that Foreman is a highly entertaining writer. His witty quips about everything from middle school dances to Target predicting teen pregnancies were a great motivator to keep me reading along, and more than once I caught myself chuckling out loud at an unexpectedly absurd reference.

It was refreshing to read a book about data science that didn't take itself seriously and added a bit of levity to an otherwise dry (interesting, but dry) subject. Even though it was lighthearted, the book was not a joke. It had an intensity to the material that was surprising given the medium through which it was presented. Spreadsheets turned out to be a great way to show how these algorithms are built up, and you can look through the columns and rows to see how each step of each calculation is performed. Conditional formatting helps guide understanding by highlighting outliers and important contrasts in the rows of data. Excel may not be the best choice for crunching hundreds of thousands of entries in an industrial-scale model, but for learning how those models actually work, I'm convinced that it was a worthy choice.

The book starts out with a little introduction that describes what you got yourself into and justifies the choice of Excel for those of us that were a bit leery. The first chapter gives a quick tour of the important parts of Excel that are going to be used throughout the book—a skim-worthy chapter. The first real chapter jumps into explaining how to build up a k-means cluster model for the highly critical task of grouping people on a middle school dance floor. Like most of the rest of the chapters, this one starts out easy, but ramps up the difficulty so that by the end we're clustering subscribers for email marketing with a dozen or so dimensions to the data.

Chapter 3 switches gears from an unsupervised to a supervised learning model with naïve Bayes for classifying tweets about Mandrill the product vs. the animal vs. the Mega Man X character. Here we can see how irreverent, but on-point Foreman is with his explanations:

Because naïve Bayes is often called "idiot's Bayes." As you'll see, you get to make lots of sloppy, idiotic assumptions about your data, and it still works! It's like the splatter-paint of AI models, and because it's so simple and easy to implement (it can be done in 50 lines of code), companies use it all the time for simple classification jobs.

Every chapter is like this and better. You never know what Foreman's going to say next, but you quickly expect it to be entertaining. Case in point, the next chapter is on optimization modeling using an example of, what else, commercial-scale orange juice mixing. It's just wild; you can't make this stuff up. Well, Foreman can make it up, it seems. The examples weren't just whimsical and funny, they were solid examples that built up throughout the chapter to show multiple levels of complexity for each model. I was constantly impressed with the instructional value of these examples, and how working through them really helped in understanding what to look for to improve the model and how to make it work.

After optimization came another dive into cluster analysis, but this time using network graphs to analyze wholesale wine purchasing data. This model was new to me, and a fascinating way to use graphs to figure out closely related nodes. The next chapter moved on to regression, both linear and non-linear varieties, and this happens to be the Target-pregnancy example. It was super interesting to see how to conform the purchasing data to a linear model and then run the regression on it to analyze the data. Foreman also had some good advice tucked away in this chapter on data vs. models:

You get more bang for your buck spending your time on selecting good data and features than models. For example, in the problem I outlined in this chapter, you'd be better served testing out possible new features like "customer ceased to buy lunch meat for fear of listeriosis" and making sure your training data was perfect than you would be testing out a neural net on your old training data.

Why? Because the phrase "garbage in, garbage out" has never been more applicable to any field than AI. No AI model is a miracle worker; it can't take terrible data and magically know how to use that data. So do your AI model a favor and give it the best and most creative features you can find.
As I've learned in the other data science books, so much of data analysis is about cleaning and munging the data. Running the model(s) doesn't take much time at all.

We're into chapter 7 now with ensemble models. This technique takes a bunch of simple, crappy models and improves their performance by putting them to a vote. The same pregnancy data was used from the last chapter, but with this different modeling approach, it's a new example. The next chapter introduces forecasting models by attempting to forecast sales for a new business in sword-smithing. This example was exceptionally good at showing the build-up from a simple exponential smoothing model to a trend-corrected model and then to a seasonally-corrected cyclic model all for forecasting sword sales.

The next chapter was on detecting outliers. In this case, the outliers were exceptionally good or exceptionally bad call center employees even though the bad employees didn't fall below any individual firing thresholds on their performance ratings. It was another excellent example to cap off a whole series of very well thought out and well executed examples. There was one more chapter on how to do some of these models in R, but I skipped it. I'm not interested in R, since I would just use Python, and this chapter seemed out of place with all the spreadsheet work in the rest of the book.

What else can I say? This book was awesome. Every example of every model was deep, involved, and appropriate for learning the ins and outs of that particular model. The writing was funny and engaging, and it was clear that Foreman put a ton of thought and energy into this book. I highly recommend it to anyone wanting to learn the inner workings of some of the standard data science models.

Python Machine Learning

This is a fairly long book, certainly longer than most books I've read recently, and a pretty thorough and detailed introduction to machine learning with Python. It's a melding of a couple other good books I've read, containing quite a few machine learning algorithms that are built up from scratch in Python a la Data Science from Scratch, and showing how to use the same algorithms with scikit-learn and TensorFlow a la the Python Data Science Handbook. The text is methodical and deliberate, describing each algorithm clearly and carefully, and giving precise explanations for how each algorithm is designed and what their trade-offs and shortcomings are.

As long as you're comfortable with linear algebraic notation, this book is a straightforward read. It's not exactly easy, but it never takes off into the stratosphere with the difficulty level. The authors also assume you already know Python, so they don't waste any time on the language, instead packing the book completely full of machine learning stuff. The shorter first chapter still does the introductory tour of what machine learning is and how to install the correct Python environment and libraries that will be used in the rest of the book. The next chapter kicks us off with our first algorithm, showing how to implement a perceptron classifier as a mathematical model, as Python code, and then using scikit-learn. This basic sequence is followed for most of the algorithms in the book, and it works well to smooth out the reader's understanding of each one. Model performance characteristics, training insights, and decisions about when to use the model are highlighted throughout the chapter.

Chapter 3 delves deeper into perceptrons by looking at different decision functions that can be used for the output of the perceptron model, and how they could be used for more things beyond just labeling each input with a specific class as described here:

In fact, there are many applications where we are not only interested in the predicted class labels, but where the estimation of the class-membership probability is particularly useful (the output of the sigmoid function prior to applying the threshold function). Logistic regression is used in weather forecasting, for example, not only to predict if it will rain on a particular day but also to report the chance of rain. Similarly, logistic regression can be used to predict the chance that a patient has a particular disease given certain symptoms, which is why logistic regression enjoys great popularity in the field of medicine.

The sigmoid function is a fundamental tool in machine learning, and it comes up again and again in the book. Midway through the chapter, they introduce three new algorithms: support vector machines (SVM), decision trees, and K-nearest neighbors. This is the first chapter where we see an odd organization of topics. It seems like the first part of the chapter really belonged with chapter 2, but including it here instead probably balanced chapter length better. Chapter length was quite even throughout the book, and there were several cases like this where topics were spliced and diced between chapters. It didn't hurt the flow much on a complete read-through, but it would likely make going back and finding things more difficult.

The next chapter switches gears and looks at how to generate good training sets with data preprocessing, and how to train a model effectively without overfitting using regularization. Regularization is a way to systematically penalize the model for assigning large weights that would lead to memorizing the training data during training. Another way to avoid overfitting is to use ensemble learning with a model like random forests, which are introduced in this chapter as well. The following chapter looks at how to do dimensionality reduction, both unsupervised with principal component analysis (PCA) and supervised with linear discriminant analysis (LDA).

Chapter 6 comes back to how to train your dragon…I mean model…by tuning the hyperparameters of the model. The hyperparameters are just the settings of the model, like what its decision function is or how fast its learning rate is. It's important during this tuning that you don't pick hyperparameters that are just best at identifying the test set, as the authors explain:

A better way of using the holdout method for model selection is to separate the data into three parts: a training set, a validation set, and a test set. The training set is used to fit the different models, and the performance on the validation set is then used for the model selection. The advantage of having a test set that the model hasn't seen before during the training and model selection steps is that we can obtain a less biased estimate of its ability to generalize to new data.

It seems odd that a separate test set isn't enough, but it's true. Training a machine isn't as simple as it looks. Anyway, the next chapter circles back to ensemble learning with a more detailed look at bagging and boosting. (Machine learning has such creative names for things, doesn't it?) I'll leave the explanations to the book and get on with the review, so the next chapter works through an extended example application to do sentiment analysis of IMDb movie reviews. It's kind of a neat trick, and it uses everything we've learned so far together in one model instead of piecemeal with little stub examples. Chapter 9 continues the example with a little web application for submitting new reviews to the model we trained in the previous chapter. The trained model will predict whether the submitted review is positive or negative. This chapter felt a bit out of place, but it was fine for showing how to use a model in a (semi-)real application.

Chapter 10 covers regression analysis in more depth with single and multiple linear and nonlinear regression. Some of this stuff has been seen in previous chapters, and indeed, the cross-referencing starts to get a bit annoying at this point. Every single time a topic comes up that's covered somewhere else, it gets a reference with the full section name attached. I'm not sure how I feel about this in general. It's nice to be reminded of things that you've read about hundreds of pages back and I've read books that are more confusing for not having done enough of this linking, but it does get tedious when the immediately preceding sections are referenced repeatedly. The next chapter is similar with a deeper look at unsupervised clustering algorithms. The new k-means algorithm is introduced, but it's compared against algorithms covered in chapter 3. This chapter also covers how we can decide if the number of clusters chosen is appropriate for the data, something that's not so easy for high-dimensional data.

Now that we're two-thirds of the way through the book, we come to the elephant in the machine learning room, the multilayer artificial neural network. These networks are built up from perceptrons with various activation functions:

However, logistic activation functions can be problematic if we have highly negative input since the output of the sigmoid function would be close to zero in this case. If the sigmoid function returns output that are close to zero, the neural network would learn very slowly and it becomes more likely that it gets trapped in the local minima during training. This is why people often prefer a hyperbolic tangent as an activation function in hidden layers.

And they're trained with various types of back-propagation. Chapter 12 shows how to implement neural networks from scratch, and chapter 13 shows how to do it with TensorFlow, where the network can end up running on the graphics card supercomputer inside your PC. Since TensorFlow is a complex beast, chapter 14 gets into the nitty gritty details of what all the pieces of code do for implementation of the handwritten digit identifier we saw in the last chapter. This is all very cool stuff, and after learning a bit about how to do the CUDA programming that's behind this library with CUDA by Example, I have a decent appreciation for what Google has done with making it as flexible, performant, and user-friendly as they can. It's not simple by any means, but it's as complex as it needs to be. Probably.

The last two chapters look at two more types of neural networks: the deep convolutional neural network (CNN) and the recurrent neural network (RNN). The CNN does the same hand-written digit classification as before, but of course does it better. The RNN is a network that's used for sequential and time-series data, and in this case, it was used in two examples. The first example was another implementation of the sentiment analyzer for IMDb movie reviews, and it ended up performing similarly to the regression classifier that we used back in chapter 8. The second example was for how to train an RNN with Shakespeare's Hamlet to generate similar text. It sounds cool, but frankly, it was pretty disappointing for the last example of the most complicated network in a machine learning book. It generated mostly garbage and was just a let-down at the end of the book.

Even though this book had a few issues, like tedious code duplication and explanations in places, the annoying cross-referencing, and the out-of-place chapter 9, it was a solid book on machine learning. I got a ton out of going through the implementations of each of the machine learning algorithms, and wherever the topics started to stray into more in-depth material, the authors provided references to the papers and textbooks that contained the necessary details. Python Machine Learning is a solid introductory text on the fundamental machine learning algorithms, both in how they work mathematically how they're implemented in Python, and how to use them with scikit-learn and TensorFlow.


Of these two books, Data Smart is a definite-read if you're at all interested in data science. It does a great job of showing how the basic data analysis algorithms work using the surprisingly effect method of laying out all of the calculations in spreadsheets, and doing it with good humor. Python Machine Learning is also worth a look if you want to delve into machine learning models, see how they would be implemented in Python, and learn how to use those same models effectively with scikit-learn and TensorFlow. It may not be the best book on the topic, but it's a solid entry and covers quite a lot of material thoroughly. I was happy with how it rounded out my knowledge of machine learning.

Realms Of Arkania: Blade Of Destiny: Summary And Rating

        

Realms of Arkania: Blade of Destiny

Germany

Released in Germany as Das Schwarze Auge: Die Schicksalsklinge

attic Entertainment Software (developer); Fantasy Productions (German publisher); Sir-Tech (U.S. Publisher)

Released 1992 for DOS, 1993 for Amiga

Date Started: 13 November 2019

Date Ended: 7 February 2020

Total Hours: 38

Difficulty: Moderate-Hard (3.5/5)

Final Rating: (To come later)

Ranking at Time of Posting: (To come later)

Summary:

First in a lineage based on the German tabletop RPG Das Schwarze Auge, Blade of Destiny is a gem waiting to be cut and polished. A party of six, comprising familiar races but original classes, stops a horde of orcs from razing the city of Thorwal by finding a legendary sword that defeated the orcs in the past. In an effort to offer a computer game that adhered closely to tabletop rules and gaming style, Blade perhaps errs too much towards obtuse statistics, lengthy character creation and leveling, myriad spells, and exhausting tactical combat. Yet the developers managed to create a large, open game world and populate it with interesting encounters of a variety of length and difficulty, thus feeling a lot like a series of tabletop modules. Nothing in the game--first-person exploration (in Bard's Tale style, but with an interface drawn from Might and Magic III), paper-doll inventories (looking a lot like Eye of the Beholder), axonometric combat (clearly inspired by SSI), dozens of skills and spell skills--works badly, but almost every part of the game needed a little tweaking, editing, or tightening. I enjoyed it more as I became more familiar with its conventions, and it left me looking forward to its next installment.

****

      

I grew to enjoy Blade of Destiny more as the hour grew later (the opposite of what usually happens), although the game never really did manage to solve some of its early weaknesses. In the end, I'm struck at how much it reminds me of Pool of Radiance, the first attempt at a serious adaptation of another tabletop system. Both feature the standard party of six. In neither game do the party members have a direct, personal connection to the main quest. In both, the main quest is somewhat low-key--the fate of a city versus the fate of the world. Both keep character leveling in the single digits, and both err towards keeping faith with their tabletop roots, even when it might have been best for the computer game to improvise a bit.

I don't know whether to blame Das Schwarze Auge or the computer game for my chief complaints, most of which can be rolled up into three words: combat is exhausting. Combat is a major part of any RPG, so you don't want your players doing things like reloading to avoid it, which I did a lot. I abandoned entire dungeons because I was sick of all the fighting, so it's a good thing I didn't need an extra character level to win. The primary issues are:

           

• The axonometric perspective doesn't work well for combat. It's hard to separate the characters and enemies from each other and particularly hard to move to a specific tile.

• Everyone misses too often.

• Attacks don't cause enough damage.

• Spells, which would make the whole thing go faster, eat up so many magic points that you can rarely cast more than three or four before needing multiple nights' rest to recharge.

         

In light of these things, the "quick combat" system was a good idea. Unfortunately, combat is hard enough (at least until the end) that you can't really use it until there are only a couple enemies left. Even then, quick combat isn't really "quick." (To be fair, I guess they don't call it that; it's something like "Computer Fight.") You still have to watch the computer take all the actions and monitor your characters' status. It just means you can watch a television show at the same time.
           

If you can make out individual characters in those blobs, your eyesight is better than mine.

         

The spell issue had more consequences than just a difficult combat experience. The developers took the time to put several dozen spells into the game, and I never used more than about 5 of them. I kept meaning to find a good place to save near a known combat and then just keep reloading and experimenting, but I never identified an ideal position for this. Most of them would have failed anyway because the nature of the spell skill system means that you can't possibly specialize in more than half a dozen. When I play the sequel, it will absolutely be my priority to more fully investigate the spell catalog.

I had a few lingering questions after the last entry, such as what happens if you try to kill the orc champion with a weapon other than Grimring, and what happens if you don't honor the rule that only your champion fights. Unfortunately, the final save prompted me to overwrite the save game I'd taken just before the battle. My next-most recent save was from before exploring the orc caves and getting the message that led to the endgame. I'm not willing to do all of that again, so we'll have to leave it a mystery unless someone has some experience with it. But I was able to check out the alternate "bad" ending, which I would have experienced had I lingered for an extra year in the quest. As I typed the rest of this entry, I had my party sleep at the inn for batches of 99 days until the game woke me up with the fateful message:
             

So the orcs are the "Vikings" of this setting.

            
Overall, I felt that the time constraint was generous enough that it wouldn't have impacted my approach even if I'd been more eager to explore every trail and sea lane. This is a good thing because there was quite a bit more to find. I took a look at a cluebook for the game, and among entire dungeons that I overlooked were a "wolf's lair" between Ottarje and Orvil, a six-level "ship of the dead" that I would have found if I'd taken more boat trips, and a three-level "dragon's hoard" on Runin Island. This latter location sounds like it would have been especially lucrative, with an option to do a side quest for the dragon and receive four magic items as a reward.

But I've always been fine with missing content. It's practically necessary in modern games, lest you exhaust yourself before the end. It also enhances a game's replayability. It's nice to see the number of titles with such optional content growing.

Let's give it the ol' GIMLET:

1. Game World. I didn't find the Nordic setting terribly original, but I enjoyed it just the same. The backstory is set up well, and as previously mentioned, I liked the low-key nature of the main plot. The main quest did a good job encouraging nonlinear exploration of the large world. The problem is that the game itself doesn't quite deliver on the backstory (or the tabletop setting in general). The various cities and towns are too interchangeable, the NPCs too bland. Score: 5.

2. Character Creation and Development. Well, I can't complain that it doesn't give you enough options. The leveling-up process in Blade of Destiny is probably the longest in any game to date. Not just longest, but most frustrating, with the caps on the number of times you can increase a particular skill per level (even if you neglected it in the early levels) and frequent failures as you try to increase. The caps in particular make it feel like the characters are never really getting stronger or better. (I think the final battle could be won by a Level 1 character.) Hit points and spell points, in particular, are almost imperceptibly slow to increase.
         

No, not now! I have an appointment in 90 minutes!

         
Still, I like the nature of character classes in the setting, including the use of "negative attributes" and the plethora of skills. I just wish I had a clearer sense of what skills, attributes, and negative attributes came into play in what circumstances, which bits of equipment compensated for them, and so on. The game text is obtuse enough that sometimes it's not even clear whether you succeeded or failed. When it is, it's almost always because you failed. Honestly, how high do I need to jack up my "Treat Wounds" skill before it has a greater than 50% chance of not making the character worse?

Back on the positive side, I think different party compositions would make a considerable difference in gameplay. I think you could have fun with some interesting combinations, like an all-dwarf party or an all-magician party. It's just too bad the different race/class templates didn't have more role-playing implications. Score: 5.

3. NPC Interaction. This was a really wasted area of the game. The developers give you the ability to talk to every bartender, innkeeper, smith, and cashier, but most of the dialogue is stupid when it isn't confusing. I'd blame the translation, but my German readers report that it was stupid and confusing even in German. The few dialogue options are either false options that lead to the same outcome or confusing ones with counter-intuitive results (e.g., asking to see the map makes the NPC give it to you; asking for the map makes him just show it to you). That said, you occasionally get an important hint from your various NPC interactions. I just wish it had been more consistent and that the developers had used the system to give more blood to the game world. Score: 4.
              

This conversation made no sense as a whole, and these individual responses made no sense in detail.

         
4. Encounters and Foes. The game shines, though sometimes with a marred finish, in this area. I really enjoyed the variety of encounters, some fixed, some random, that the party gets on the road and as it explores dungeons and towns. I like that some of them are a single screen, resolved instantly, and others lead you off on a multi-hour digression. In contrast to the dialogue, the text of these special encounters is usually evocative and interesting, and I can even forgive the occasional shaggy dog joke like the "wyvern" encounter. I just wish for a few more role-playing options in these encounters.
           

These diversions and side areas never stopped being fun.

           
Foes were mostly high-fantasy standards with similar strengths and weaknesses that we've seen in a thousand RPGs but at least they appeared in appropriate contexts. We've come a long way from the days when we were inexplicably attacked by parties of 6 orcs, 3 trolls, 2 magicians, and a griffon right in the middle of town. Score: 6.

5. Magic and Combat. Very mixed. I like the combat options, the variety of spells, and the turn-based mechanics. I just didn't like the execution, which was partly due to interface and partly due to the game rules. Either way, combat was generally a tedious, annoying process rather than the joyful one I typically find in, say, a Gold Box game. As for spells, the game really needs some in-game help to assist with them, perhaps annotating the spells in which each class is supposed to specialize. Every spellcasting session and every level-up was a long process of flipping through the manual. It's too bad because the spells are so varied and interesting on paper. Score: 4.
           

I only ever tried about 6 of these spells, which coincidentally is the number of spells I got above 0 in my ability to cast after 5 levels.

         
6. Equipment. Another disappointment. I like the approach to equipment, with a number of slots, but you get upgrades rarely and it's extremely hard to identify them when you do. This is something that perhaps no game has done very well up to this point. I don't mind if it's hard to identify a piece of equipment--if you need a special skill, or spell, or money, or whatever--but I mind if it's annoying. I mind if I have to swap the item around to multiple characters to try different things, especially when the interface makes swapping annoying and time-consuming. I mind when there's no symbol, color, or other mechanism to distinguish weapons and armor with different values. 

Blade offers perhaps the largest variety of "adventure" equipment that we've seen so far, which makes it all the more frustrating that either so much of it is useless, or the game doesn't bother to tell you when a piece of equipment has saved the day. Finally the encumbrance system is geared towards making most characters chronically over-encumbered. The ability to make potions is nice, but again the system is a little too complicated. Score: 4.

7. Economy. Blade almost perfectly emulates the Gold Box series here: money is plentiful from the first dungeon and you hardly have any reason to spend it. My party ended the game with well over 1,000 ducats. Even potions don't serve as a good "money sink" because they don't stack and you have the constant encumbrance issue. A rack of +1 weapons, the ability to pay to recharge spell points, or temple blessings that actually did something all would have been nice. Score: 3.
         

I just donated 999 crowns!

         
8. Quests. Generally positive. Blade is one of the few games of the era to understand side quests, and they sit alongside an interesting-enough main quest with multiple stages. It just needed a few more choices and alternate endings. Score: 5.

9. Graphics, Sound, and Interface. I know that some readers will defend the game here, but I found all three to be somewhat horrid. Graphics are perhaps the least so. Some of the cut scenes are nice. Regular exploration graphics aren't bad, but the inability to distinguish stores from regular houses is almost unforgivable. Combat graphics are a confusing mess from the axonometric perspective. Any virtues the sound effects may otherwise have are obscured by the jarring three-note cacophony that accompanies opening any menu. And there's no excuse for the interface, which occasionally gives some nods to the keyboard but really wants you to use the mouse throughout.

Aside from my usual complaints about mouse-driven interfaces, the game is full of all kinds of little annoyances. When you find or purchase a piece of equipment, it always goes to the first character. You've got to then go in and redistribute it. It's annoying to transfer equipment between characters, especially if one is over-encumbered. Messages often time out before you're done reading them, or pop up so quickly that you don't have time to read them before you accidentally hit the next movement key, making them disappear. There's a lot of inconsistency, particularly in dungeons, about when you need a contextual menu and when you need to use the buttons on the main interface. There are dozens of other things like this. The developers took the appearance of the Might and Magic III interface but none of its underlying grace.

The auto-map didn't suck. I'll give it that. Score: 2.

10. Gameplay. We can end on a positive note. This is one of the few open-world games of the era, and in between the opening screen and closing combat, it's almost entirely non-linear. The many things that a first-time player doesn't find makes it inherently replayable. And the length and difficulty are just about perfect for the era. I particularly love that you have to lose experience points to save (except at temples), which discourages save-scumming. Score: 8.
               

This NPC seems to think he's living hundreds of years in the past.

         
That gives us a subtotal final score of 46, a respectable total that would put it in the top 15% of games so far. But I'm going to administratively remove 2 more points for an issue that really isn't covered by my GIMLET: a lack of editing that created unnecessary confusion at numerous points in the game. There are numerous places that go unused, such as the tower and "Ottaskins" in Thorwal. NPCs frequently tell you things in dialogue that aren't true. There are numerous false leads on the map quest, and I don't think they're there to challenge you--I think the developers changed things and didn't update the dialogue. All of the NPCs in Phexcaer were clearly written for an earlier game in which the nature of the backstory and quest were quite different. It's common now, but relatively uncommon back then, to find a game released in what was clearly its "beta" stage.

So that gives us a final score of 44, which still puts the game in the top 15%. It had a lot of promise, and I'm sorry that the developers didn't find more time to tweak and tighten it.
            

This is not the sort of game for which you really want to emphasize "conversation."

         

Blade of Destiny wasn't released in the United States until 1993, so Scorpia didn't take it on until the October 1993 issue of Computer Gaming World. It's one of her more ornery reviews. After saying that the English translation of Das Schwarze Auge, "The Black Eye," "might be appropriate," she goes on to spoil the entire plot in the next paragraph, including the one-on-one combat at the end. She found the plot unoriginal and wasted three days trying to figure out how to find the orc cave, noting that there are no clues to be found anywhere. (Remember: I had to use a walkthrough for this.) She hated the failures when trying to level up, complaining that one of her fighters "made no advance in swords on two successive level gains." She noted a lot of discrepancies between the manual and actual gameplay, particularly in the area of spells, and she agrees with me that combat is a "tedious, frustrating, boring, long-drawn-out affair."

She liked the automap, the ability to reload in the middle of combat, and the extra experience you get the first time you face a particular monster. That was about it. I was surprised to see how much she hated the experience cost for saving. She says she wouldn't have minded if the creators had awarded a bonus for not saving, apparently seeing a difference there that I don't. 

But her worst vitriol was for a bug that I didn't experience: apparently, if you quit in the middle of the final battle, you get the victory screen anyway. "This is not just a scam; it is the Grand Canyon of scams," she sputters. "How did the 20+ playtesters manage to miss this one? If they didn't miss it, why wasn't it fixed?" In summary:

            

Those who worship at the mythical altar of Realism often end up sacrificing fun and playability on it. That is what happened with Blade of Destiny. In their attempt to make the game "like real life" (something few players want in the first place) the designers went overboard in the wrong direction more than once. I would not recommend Arkania to any game player, but I do recommend it to game designers as an example of what to avoid in their own products. Let us all hope we don't see another one like this any time soon.

             

Ouch. I don't disagree with the elements she didn't like, but I found more that I did like.

On the continent, the game had polarized reviews. Some thought that the designers went overboard in the right direction, or perhaps didn't go overboard, or perhaps only did it once. Whatever the case, the ASM reviewer (92/100) said that he'd "rarely seen a perfect implementation of an RPG that also remains really playable on the computer." PC Joker (90/100) said that it is "only surpassed by Ultima, leaving the rest of the genre competition far behind in terms of freedom of action and complexity." But not all German reviews were positive. PC Player (48/100) recommended that players "close your eyes, put the lid on, and wait for Star Trail."

(At least there were some positives in the reviews for the original game. A 2013 remake by German-based Crafty Studios came out to almost universally negative reviews despite improved graphics, voiced dialogue, and other trappings of the modern era. It was apparently quite unforgivably bugged. Crafty went on to remake Star Trail in 2017.)

             

Combat in the remake. At least you can identify the squares a bit easier.

         

The original game sold well despite a few bad reviews and certainly justified the two sequels, Realms of Arkania: Star Trail (1994) and Realms of Arkania: Shadows over Riva (1996). Together, the trilogy established the viability of Das Schwarze Auge setting, which continues to produce RPGs into the modern era, including The Dark Eye: Drakensang (2008), Deminicon (2013), and Blackguards (2014). Lead developer Guido Henkel would eventually tire of the setting, quit attic, move to the United States, join Black Isle studios, and produce Planescape: Torment.

I haven't attempted to reach out to Henkel, as his work on the Arkania series has been well-documented elsewhere. In his 2012 RPG Codex interview, he explains that the publisher of attic's Spirit of Adventure, StarByte, originally approached the company about creating a series based on Das Schwarze Auge, claiming they already had the rights. The attic personnel were reluctant to work with StarByte after a dreadful Spirit experience ("a horribly crooked company that cheated us and all of its other developers"), so they were delighted to find that the company had been lying about the license. attic managed to get it for themselves, although at such an expense that the three Arkania games barely made a profit despite selling well.

From a 1992 perspective, I would call Blade of Destiny "a good start." I look forward to seeing how things change in the sequels.

*****

B.A.T. II will be coming up next. For the next title on the "upcoming" list, we reach back to 1981 for Quest for Power, later renamed King Arthur's Heir. Come to think of it, the Crystalware titles are so similar and quick that I might try to cover Quest for Power and Sands of Mars in a single session so I can be done with 1981 entirely. Again.

Falcons, Spears And Revenants, Oh My

I finished the first of the six Falcon turrets last Friday right before NEAT. The next six are going to be magnetised so I can swap them out for Fire Prism turrets (haven't started on those yet though).

I also did some conversion work earlier this week on my Revenants. I did some leg swaps to add some pose variety and magnetized the weapons. I have some Vibrocannon arm conversions to work on and a bit of puttying to do before these see paint.

Finally, I finished up the lances and gems on the Shining Spears last night, and put on their decals as well. They'll need a bit of touch up and bases edges before they get top coated.

PlayStation Won'T Attend PAX East Over Coronavirus Fears - Eurogamer

PlayStation won't attend PAX East over coronavirus fears

Uclan Games Design Students In Global Games Jam 2020!

What a truly awesome bunch of students and Alumni we have!
They've all worked really hard together in groups over the whole weekend for long hours and they've now completed the Global Game Jam 2020.

The Global Game Jam 2020 started on Fri 31st Jan in all sites around the world, and this year's theme is...

Repair
The theme reveal clip at the end of the GGJ20 keynote video showed objects in various states of repair, breaking and repairing buildings and pots, repairing relationships, and plants repairing themselves.

See the theme video.

You can see the UCLan Game Jam link here.

https://globalgamejam.org/2020/jam-sites/university-central-lancashire

Lewis Wright, who organised the event at UCLan with Simon Ashcroft said, 'Everyone smashed it out of the park, cleanest game jam I've been a part of. Every group had fun and a working game by the end of it.'

They recorded all of the games that were shown at the end, including the game dev ones as a comparison. They're gonna make a show-reel out of them for us all to see on the YouTube channel.

They even tidied the studio room before they left. Time for some well deserved sleep for them all. Wish I could have joined them this year as the comradery and sense of fun was evident as always as we watched them working away via Twitch TV. ☺️

So proud of you all!

Nonsense And Other Observations About Dracula/Horror Of Dracula

As much as I love Dracula (1958)/Horror of Dracula, I find it has a lot of plot holes, ideas that don't make a lot of sense and other weirdness.  I started to write down certain observations as I was watching the film and eventually they became so long that I thought them worth putting them up in a blog post.  These observations as I have put them to type have been as timestamped to the times (roughly) in the film to which they most apply.  I am using the 2012 Hammer Restoration found on the 2013 Region B/2 Blu-ray/DVD from Lionsgate with altered color timing by a fan (which eliminates the overly-blue tint found on that disc).  Any DVD or Blu-ray release should be able to follow along without too much difficulty.

Read more »

You say "obsessed" as if it is a bad thing.

Tech Book Face Off: Effective Python Vs. Data Science From Scratch

I must confess, I've used Python for quite some time without really learning most of the language. It's my go-to language for modeling embedded systems problems and doing data analysis, but I've picked up the language mostly through googling what I need and reading the abbreviated introductions of Python data science books. It was time to remedy that situation with the first book in this face-off: Effective Python: 59 Specific Ways to Write Better Python by Brett Slatkin. I didn't want a straight learn-a-programming-language book for this exercise because I already knew the basics and just wanted more depth. For the second book, I wanted to explore how machine learning libraries are actually implemented, so I picked up Data Science from Scratch: First Principles with Python by Joel Grus. These books don't seem directly related other than that they both use Python, but they are both books that look into how to use Python to write programs in an idiomatic way. Effective Python focuses more on the idiomatic part, and Data Science from Scratch focuses more on the writing programs part.

VS.

Effective Python

I thought I had learned a decent amount of Python already, but this book shows that Python is much more than list comprehensions and remembering self everywhere inside classes. My prior knowledge on the subjects in the first couple chapters was fifty-fifty at best, and it went down from there. Slatkin packed this book with useful information and advice on how to use Python to its fullest potential, and it is worthwhile for anyone with only basic knowledge of the language to read through it.

The book is split into eight chapters with the title's 59 Python tips grouped into logical topics. The first chapter covers the basic syntax and library functions that anyone who has used the language for more than a few weeks will know, but the advice on how to best use these building blocks is where the book is most helpful. Things like avoiding using start, end, and stride all at once in slices or using enumerate instead of range are good recommendations that will make your Python code much cleaner and more understandable.

Sometimes the advice gets a bit far-fetched, though. For example when recommending to spell out the process of setting default function arguments, Slatkin proposed this method:

def get_first_int(values, key, default=0):
    found = values.get(key, [''])
    if found[0]:
        found = int(found[0])
    else:
        found = default
    return found

Over this possibility using the or operator short-circuit behavior:

def get_first_int(values, key, default=0):
    found = values.get(key, [''])[0]
    return int(found or default)

He claimed that the first was more understandable, but I just found it more verbose. I actually prefer the second version. This example was the exception, though. I agreed and was impressed with nearly all of the rest of his advice.

The second chapter covered all things functions, including how to write generators and enforce keyword-only arguments. The next chapter, logically, moved into classes and inheritance, followed by metaclasses and attributes in the fourth chapter. What I liked about the items in these chapters was that Slatkin assumes the reader already knows the basic syntax so he spends his time describing how to use the more advanced features of Python most effectively. His advice is clear and direct so it's easy to follow and put to use.

Next up is chapter 5 on concurrency and parallelism. This chapter was great for understanding when to use threads, processes, and the other concurrency features of Python. It turns out that threads and processes have unique behavior (beyond processes just being heavier weight threads) because of the global interpreter lock (GIL):

The GIL has an important negative side effect. With programs written in languages like C++ or Java, having multiple threads of execution means your program could utilize multiple CPU cores at the same time. Although Python supports multiple threads of execution, the GIL causes only one of them to make forward progress at a time. This means that when you reach for threads to do parallel computation and speed up your Python programs, you will be sorely disappointed.

If you want to get true parallelism out of Python, you have to use processes or futures. Good to know. Even though this chapter was fairly short, it was full of useful advice like this, and it was possibly the most interesting part of the book.

The next chapter covered built-in modules, and specifically how to use some of the more complex parts of the standard library, like how to define decorators with functools.wraps, how to make some sense of datetime and time zones, and how to get precision right with decimal. Maybe these aren't the most interesting of topics, but they're necessary to get right.

Chapter 7 covers how to structure and document Python modules properly when you're collaborating with the rest of the community. These things probably aren't useful to everyone, but for those programmers working on open source libraries it's helpful to adhere to common conventions. The last chapter wraps up with advice for developing, debugging, and testing production level code. Since Python is a dynamic language with no static type checking, it's imperative to test any code you write. Slatkin relates a story about how one programmer he knew swore off ever using Python again because of a SyntaxError exception that was raised in a running production program, and he had this to say about it:

But I have to wonder, why wasn't the code tested before the program was deployed to production? Type safety isn't everything. You should always test your code, regardless of what language it's written in. However, I'll admit that the big difference between Python and many other languages is that the only way to have any confidence in a Python program is by writing tests. There is no veil of static type checking to make you feel safe.

I would have to agree. Every program needs to be tested because syntax errors should definitely be caught before releasing to production, and type errors are a small subset of all runtime errors that can occur in a program. If I was depending on the compiler to catch all of the bugs in my programs, I would have a heckuva lot more bugs causing problems in production. Not having a compiler to catch certain classes of errors shouldn't be a reason to give up the big productivity benefits of working in a dynamic language like Python.

I thoroughly enjoyed learning how to write better Python programs through the collection of pro tips in this book. Each tip was focused, relevant, and clear, and they all add up to a great advanced level book on Python. Even better, the next time I need to remember how to do concurrency or parallelism or how to write a proper function with keyword arguments, I'll know exactly where to look. If you want to learn how to write Python code the Pythonic way, I'd highly recommend reading through this book.

Data Science from Scratch

I didn't expect to enjoy this book quite as much as I did. I went into it expecting to learn about how to implement the fundamental tools of the trade for data science, and that was indeed what I got out of the book. But I also got a lighthearted, entertaining, and surprisingly easy-to-read tour of the basics of machine learning using Python. Joel Grus has a matter-of-fact writing style and a dry wit that I immediately took to and thoroughly enjoyed. These qualities made a potentially complex and confusing topic much easier to understand, and humorous to boot, like having an excellent tour guide in a museum that can explain medieval culture in detail while cracking jokes about how toilet paper wasn't invented until the 1850s.

Of course, like so many programming books, this book starts off with a primer on the Python language. I skipped this chapter and the next on drawing graphs, since I've had just about enough of language primers by now, especially for languages that I kind of already know. The real "from scratch" parts of the book start with chapter 4 on linear algebra, where Grus establishes the basic functions necessary for doing computations on vectors and matrices. The functions and classes shown throughout the book are well worth typing out in your own Python notebook or project folder and running through an interpreter, since they are constantly being used to build up tooling in later chapters from the more fundamental tools developed in earlier chapters. The progression of development from this chapter on linear algebra all the way to the end was excellent, and it flowed smoothly and logically over the course of the book.

The next few chapters were on statistics, probability, and their use with hypothesis testing and inference. Sometimes Grus glossed over important points here, like when explaining standard deviations he failed to mention that this metric only applies to (or at least applies best to) normal distributions. Distributions that deviate too much from the normal curve will not have meaningful standard deviations. I'm willing to cut him some slack, though, because he is covering things quickly and makes it clear that his goal is to show roughly what all of this stuff looks like in simple Python code, not to make everything rigorous and perfect. For instance, here's his gentle reminder on method in the probability chapter:

One could, were one so inclined, get really deep into the philosophy of what probability theory means. (This is best done over beers.) We won't be doing that.

He finishes up the introductory groundwork with a chapter on gradient descent, which is used extensively in the later machine learning algorithms. Then there are a couple chapters on gathering, cleaning, and munging data. He has some opinions about some API authors choice of data format:

Sometimes an API provider hates you and only provides responses in XML.

And he has some good expectation setting for the beginner data scientist:

After you've identified the questions you're trying to answer and have gotten your hands on some data, you might be tempted to dive in and immediately start building models and getting answers. But you should resist this urge. Your first step should be to explore your data.

Data is never exactly in the form that you need to do what you want to do with it, so while the gathering and the munging is tedious, it's a necessary skill that separates the great data scientist from the merely mediocre. Once we're done learning how to whip our data into shape, it's off to the races, which is great because we're now halfway through this book.

The chapters on machine learning models, starting with chapter 12, are excellent. While Grus does not go into intricate detail on how to make the fastest, most efficient MLMs (machine learning models, not multi-level marketing), that is not the point. His objective is to show as clearly as possible what each of these algorithms looks like and that it is possible to understand how they work when shown in their essence. The models include k-nearest neighbors, naive bayes, linear regression, multiple regression, logistic regression, decision trees, neural networks, and clustering. Each of these models is actually conceptually simple, and the models can be described in dozens of lines of code or less. These implementations may be doggedly slow for large data sets, but they're great for understanding the underlying ideas of each algorithm.

Threaded through each of these chapters are examples of how to use each of the statistical and machine learning tools that is being developed. These examples are presented within the context of the tasks given to a new data scientist who is an employee of a budding social media startup for…well…data scientists. I just have to say that it is truly amazing how many VPs a young startup can support, and I feel awfully sorry for this stalwart data scientist fulfilling all of their requests. This silliness definitely keeps the book moving along.

The next few chapters delve a bit deeper into some interesting problems in data science: natural language processing, network analysis (or graph algorithms), and recommender systems. These chapters were just as great as the others, and by now we've built up our data science tooling pretty well from the original basics of linear algebra and statistics. The one thing we haven't really talked about, yet, is databases. That's the topic of the 23rd chapter, where we implement some of the basic operations of SQL in Python in the most naive way possible. Once again it's surprising to see how little code is needed to implement things like SELECT or INNER JOIN as long as we don't give a flying hoot about performance.

Grus wraps things up with an explanation of the great and all-powerfull MapReduce, and shows the basics of how it would be implemented with mapper and reducer functions and the plumbing to string it together. He does not get into how to distribute this implementation to a compute cluster, but that's the topic of other more complicated books. This one's done from scratch so like everything else, it's just the basics. That was all fine with me because the basics are really important, and knowing the basics well can lead you to a much deeper understanding of the more complex concepts much faster than if you were to try to dive into the deep end without knowing the basic strokes. This book provides that foundation, and it does it with flair. I highly recommend giving it a read.


Both Effective Python and Data Science from Scratch were excellent books, and together they could give a programmer a solid foundation in Python and data science as long as they already have some experience in the language. With that being said, Data Science from Scratch will not provide the knowledge on how to use the powerful data analysis and machine learning libraries like numpy, pandas, scikit-learn, and tensorflow. For that, you'll have to look elsewhere, but the advanced, idiomatic Python and fundamental data science principles are well covered between these two books.