Damned Lies and Statistics

It’s a poorly-kept secret that computer games cheat. What’s unfortunate is when the developers try to deny that is what they do to increase the apparent strength of an AI opponent.

Obviously this is something that isn’t applicable to all games. Chess or Go have no way to cheat. However there are a lot of games with endless opportunities for the game’s developer to use dirty tricks to make their AI players more formidable.

One game that I play a lot on my smart phone is Backgammon by AI Factory. It’s a free game with a reasonable user interface, but the developers couldn’t even program a reasonable algorithm for “bearing off” (removing pieces after you’ve got them all into your home quarter).

The basic idea is to optimize the removal of pieces for each dice roll. If you’ve got all 6 spaces covered in your home quarter, you remove a piece from each number rolled on the two dice. That’s simple enough.

Where it gets trickier is when you don’t have all the spaces covered. The basic tactic is to remove one piece if possible. However you also want to make the best use of the numbers rolled on the two dice by moving other pieces closer to the end.

This is where the AI falls down. Say you rolled a 6 and a 1 and your highest remaining piece was 5 away from the end. A rational person would use the 6 to remove the 5 piece and use the 1 to move another piece closer to the end. The AI will use the 1 first to move the 5 piece to 4 then use the 6 to bear it off. This wasted 2 out of the 7 rolled when an optimum tactic would only have wasted 1.

It does this because there are situations where using the smaller number first makes sense. However their algorithm is basically wrong. In a close game, letting the AI handle the tedious task of bearing off could result in you losing. Conversely, because it controls your AI opponent, it also means that the AI loses more games that it should have based on the dice roles.

Faced with an AI that plays an inferior game, the developers resorted to the old standby tactic of cheating. However they don’t want to admit that’s what they do, so to back up their case, they collect statistics on the games you’ve played.

According to them, I’ve won 4,411 matches versus the AI’s 2,419. While I’ve rolled 24,907 doubles, the AI has rolled 25,176 – a negligible difference. Similarly we’ve won almost exactly the same number of First Dice rolls.

Those are the easy statistics to gather and the easiest to verify. However any backgammon player will tell you that they are also pretty crude indicators. Slightly better are the post-game statistics on the totals of all your dice rolls.

Again though, timing is everything. If you have a piece on the bar, rolling double-sixes is unlikely to help you.

And that’s where the game doesn’t collect statistics. One consistent thing I’ve noticed is that it’s almost impossible to keep the AI’s pieces on the bar while my own pieces have extreme difficulty getting off the bar. Statistics on that could be collected but aren’t.

And double-sixes are exactly what I usually roll when I have a piece on the bar. Conversely double sixes are a common roll for the AI after I’ve decided to run for home. And they are also a common roll for my last before the AI bears its last piece off, defeating me. When it gives me double-sixes that don’t help, it allows it to produce statistics showing the dice rolls were closer than they were. When it gives itself double-sixes, I might get some other double that isn’t helpful but which allows the program to report similar numbers of doubles.

I could collect the statistics to demonstrate this but what’s the point? The game’s developers don’t appear to interested in developing a better AI, which is what I really want – an opponent that can challenge me without cheating.

I just wish they wouldn’t try to cover up their cheating behind meaningless statistics.

Posted in Science and Mathematics | Leave a comment

9/11 conspiracy theories are all delusional

I’m still seeing people posting 9/11 conspiracy theories on Facebook. When I point out that they are giving even conspiracy theorists a bad name, someone pointed me to a 9/11 conspiracy theory site, as if that somehow proved something.

I hate to break it to people but there is simply no way any of 9/11 conspiracy theories (except that it was planned and executed by Al Qaeda) could possibly be true. You can tell that because:
1) they all require too many people to keep a secret about something horrific being plotted against their own people, and
2) they all involve sufficient complexity to give Rube Goldberg nightmares.

The simplest theory that fits all the known facts is that as many as 19 Al Qaeda operatives hijacked 4 airplanes and attempted to fly them into buildings. 3 attempts succeeded while the fourth failed.

A conspiracy of this size is difficult to pull off. It only succeeded (partially) because none of the attackers were Americans. They saw the U.S. as the enemy and thus had few qualms about attacking even mainly civilian targets.

A false flag operation on the other hand would require at least similar number of people who would be willing to sacrifice their own lives to attack their fellow countrymen for nebulous political reasons.

Or perhaps the false flag involved real attackers but the planning was directed by people pretending to represent Al Qaeda. That would require fewer people to be directly involved but would require those people to be able to convince others that they were actually legitimate Al Qaeda operatives for an extended period.

Setting up a fake Al Qaeda network would be tricky but is possibly doable. However in the post-Snowden, Post-Manning and Wikileaks era, you’d expect to see some sign of it in the released documents. There isn’t any.

However the conspiracy theorists don’t stop there. They insist that planes that were seen flying into the twin towers didn’t do sufficient damage to destroy them. Bizarrely they also claim that the plane seen flying into the pentagon didn’t and that it was actually a missile.

This is where the 9/11 conspiracies venture into the delusional. What political goal could be achieved by destroying the twin towers that wasn’t already achieved by damaging them while killing hundreds of people? Absent a critical motive, why risk the entire conspiracy by adding another one to ensure that the towers came down?

And why fire a missile at the pentagon when you’ve got a flight full of passengers to account for? While it’s conceivable that a few people could be duped into hijacking a plane, arranging for 64 people to vanish is a much larger undertaking.

There’s no doubt that there was a 9/11 conspiracy. However the only one that fits the evidence without requiring a huge suspension of disbelief is that it was plotted and carried out under the direction of Al Qaeda.

Posted in Economics, History, Politics, Religion | Leave a comment

Computers can be very frustrating

A year or two back, I had a problem with a computer that I use as a file & print server. It had 6 SATA ports on the motherboard, which I used as follows:
0) boot drive (SSD)
1) BluRay writer
2 – 5) RAID 6 array for data and backups
(for those not familiar with the term, RAID 6 is a method of grouping drives so that they work together as a single large drive, with built in redundancy so that any 2 drives can fail and the data is still intact).

For some reason port 4 & 5 stopped detecting the disk drives attached to them. I solved this by purchasing a PCIe SATA card with 2 internal ports and connected the drives to it instead. This worked until a month ago. After upgrading the operating system (from Debian/Jessie (8) to Debian/Stretch (9)) the add-in card stopped being able to detect the 2 drives attached to it.

After much fiddling about, I discovered that there was nothing apparently wrong with the card or the drives. For some reason, only the first 4 ports (0- 3) could now recognize the hard disk drives, while the last 2 (4 & 5) and the add-in card could recognize the boot drive and the BluRay writer but not a hard drive. Switching the cables around and removing the add-in SATA card and I was back in business.

This lasted for almost 2 weeks. Then the computer began emitting intermittent (but frequent) beeps. It also stopped recognizing two of the drives. I eventually traced the beeping to a flaky power supply (replacing it with a new one cured the beeping) but I had to reinstall the add-in card to get all 4 hard drives recognized.

More bizarrely, I couldn’t boot into the latest version of the Linux kernel. I had to boot into the older (Debian/Jessie) kernel, which fortunately was still an option. While not ideal, I was willing to live with it. My server was still doing what it was supposed to do.

That didn’t last long. About a week ago, you guessed it, the two drives attached to the add-in card stopped being recognized.

At this point I concluded that maybe it was time to replace the motherboard. I found a cheap board that could to the job for about $100 (including processor) although I’d have to buy new RAM (the old system used DDR2 while the new board used DDR3) and a 4-port SATA card. Still it seemed like the cheapest solution so I placed the order on canadacomputers.com and asked for the free delivery to my closest store.

This was just before the long weekend. On Tuesday I got a call saying that the board I’d ordered was actually now out of stock.

Time to go for my first instinct – replace the server’s CPU, motherboard and memory with the one in my workstation. It’s been very reliable but is showing its age as a desktop system. However it’s half the age of the server components and there’s the AMD Ryzen processors now on the market. I normally buy AMD because I don’t want Intel to have a real monopoly on the desktop market.

As an aside, it’s interesting to note that with the AM4 socket, AMD has actually reunited its two socket types. Previously they’d used the AM sockets for processors without an onchip graphics processor while they’d used the FM sockets for processors with onchip graphics. Newer A-series and Athlon processors now use the same AM4 socket as the Ryzen processors.

This leads to an interesting confusion. Socket AM4 motherboards all have back-panel video ports but you won’t get a video signal from them unless you are using a processor with an onchip graphics processor. I haven’t read much discussion of this anywhere and it’s barely mentioned in the advertising.

Wednesday I bought a new CPU, motherboard and memory (since the Ryzen processors require DDR4) for my workstation. Returning home, I removed the motherboard from my server and put it aside. Then I transferred the motherboard from my workstation to the server, removing the video card though and replacing it with an more mundane one that uses passive cooling since the server doesn’t run a GUI.

I reconnected everything and started it up. This being Linux, everything started up perfectly – almost. One of the disk drives wasn’t being seen and the network wasn’t starting. This latter point is not good for a server!

Since I didn’t need the network until I got my workstation running, I first added the working disk drive back into the RAID 6 array so that there would still be some redundancy in case a drive failed. While the drive was resynchronizing with array, I started rebuilding the workstation.

I normally put the motherboard in the case before adding the CPU because it makes it easier to access all the screw locations. After getting everything connected, I went to install the CPU.

All the CPU coolers I’ve dealt with recently use a clip to hold them to the motherboard/CPU. The Ryzen’s cooler uses spring-loaded screws instead. This meant removing the motherboard’s clips so that the cooler could screw directly into the plate under the motherboard. However this plate isn’t attached to the board – it relies on the screws to hold it in place. When I removed the clips, the plate promptly fell off.

Not a big deal – I just had to remove the back of the case and hold the plate in place with one hand while trying to screw the CPU cooler in place with the other. And yes, that is as awkward as it sounds.

Finally, I inserted the memory, reconnected the peripherals, and powered it up.

Nothing. The lights came on, the fans started spinning but no sound, no video, nothing.

Back to basics. I removed everything except the video card and memory. No disk drives connected, no front USB or audio connections. Just the motherboard, memory and speaker. Still nothing. Next I removed the memory, since this should be a sure way to get the motherboard to start beeping. Another failure.

Next I shut down my server and yanked the video card from it to install in my workstation instead of the heavy-duty card. I still got nothing from the video.

After replacing the card back into the server and starting it up, I removed the motherboard from the workstation and connected it to a different power supply. I still couldn’t get the system to give a beep.

Admitting defeat, I returned to Canada Computers to give their tech support a try. I left everything with them and returned home to work on debugging the server’s missing drive issue.

By swapping cables around, I eventually found that the drive that wasn’t being recognized was apparently dead. I could attach it to the power and data cables from one of the working drives but it still wasn’t being seen.

Just then Canada Computers called. They couldn’t get the motherboard to beep either. Returning, I got a refund on the CPU and motherboard but decided to keep the memory. I then bought another CPU (same model) and a different, more expensive, motherboard, along with a replacement drive for my server.

I replaced the dead drive with the new one and restarted the server. The drive was being recognized correctly so I added it back into the RAID 6 array so I’d have the full 2-drive redundancy again. While it was rebuilding, I started rebuilding my workstation.

Learning from my mistake, this time I installed the CPU before putting the motherboard in the case. Also, I went with a minimal install, just connecting the power and switches before trying to power up the system.

Still no beeps. Perhaps the speaker was faulty? I put the video card back in and connected it to my monitor and tried again. Still nothing. I again removed the memory, since that should surely trigger some angry beeping. Utter silence.

At this point I’m wondering “WTF am I doing wrong?” One bad board could be a manufacturing defect, but two in a row is almost unheard of.

Back to Canada Computers again. When their tech guy came back, he rechecked everything then suggested perhaps it’s the memory. Apparently the Rizen processors are somewhat finicky. Moreover, unlike most processors, they don’t seem to trigger “no memory” POST warning beeps.

He grabbed a box of DDR4 RAM from another manufacturer and an open-box video card, plugged things in and powered it up. Success! Despite the new memory having identical specs to the old, the new memory worked while the original memory didn’t.

I returned home and commenced rebuilding the workstation. First I redid the minimal install, with just enough connected to get the system to boot. It worked, so I shut things down and reconnected everything. When I started it up, everything worked flawlessly – almost.

Remember, my server wasn’t starting the network properly so I didn’t have my network shares or printers available. Time to get that fixed.

I had vague recollections of network problems with the motherboard when I first installed it. I couldn’t recall how I fixed it however, so I powered up my laptop and looked for solutions online.

The onboard NIC is a Realtek R8168. This is actually a family of devices that aren’t always supported on Linux out of the box. Realtek make their own Linux driver so I went to their web site, downloaded it onto a USB stick and installed it on the server.

When this didn’t work, I decided to do what I should have done in the first place – check out what the exact failure is.

The problem seems to be that with SystemD replacing the venerable SysVinit, network interface names have changed. The old eth0, eth1, etc. only stick around if they were used before the upgrade. Since the network interface is on the replacement motherboard, it was given a new name. The old eth0 interface was tied to the old motherboard’s NIC so it is now obsolete. I had a udev persistent rule to name that old NIC eth0, so I removed it.

The new network interface is named enp5s0. However simply creating an /etc/network/interfaces file asking for enp5s0 to get its address via dhcp didn’t work. This wasn’t a good solution anyway, but I thought it would be worth a shot just to see if the network would actually work.

Since the Realtek r8168 driver didn’t seem to be working, I tried replacing it with Debian’s r8168-dkms driver. That didn’t work either. However I knew that the NIC was working when it was part of my workstation, so perhaps I had the right driver originally after all.

I removed the r8168-dkms driver and tried the r8169. This seemed to work, but I was getting an error message about firmware. Ignoring this for the moment, I returned to the /etc/network/interfaces file.

Because I run virtual machines on the server, I require network bridging. The actual NIC (enp5s0) is tied to a bridge service (br0) so I had to replace eth0 in the interfaces file with enp5s0 in both iface line and also in the bridge definition.

Another problem was, because of my earlier fiddling with the drivers, the r8169 driver was no longer loading automatically. Adding r8169 to the /etc/modules file fixed this. I also found a blacklist file set up by the r8168-dkms driver installation that I removed.

Now there was just the pesky firmware message to deal with. While it doesn’t seem to be necessary, installing the firmware-realtek package did remove the message since the package contains the correct firmware for the variant on my board.

I now have a speedy workstation and what seems to be a reliable server. Keeping my fingers crossed that no new problems will crop up!

Posted in Home repair, Internet, Science and Mathematics | Leave a comment

Good enough?

I had a bizarre discussion recently on Facebook with some people who didn’t seem to understand the difference between “good enough” and “not good enough”. They also didn’t seem to understand even the basics of any kind of system design.

Their position was that “good enough” isn’t “good enough” (a logical contradiction) and that we should always want something better. They however didn’t seem to apply this to their own decisions. They just wanted everyone else to do it.

For example, I drive a Toyota Corolla. There are better cars on the market but this one is “good enough” for what I use it for. It’s not that I settled for an inferior car. A Rolls Royce or Lamborghini simply wouldn’t be appropriate and there was no need even for me to spend more money for a Camry. I’d rather save my money for other things I want.

That’s a normal trade-off in any system. You have limited resources and have to allocate them to things that matter in order to maximize your overall satisfaction.

They replied with the argument that a bridge that was built to be “good enough” would fall down, killing them. Clearly such a bridge wasn’t “good enough” since it fell down. Things that are “good enough” don’t fail with catastrophic results. What they described was something that clearly wasn’t built or maintained “good enough”.

Similarly we don’t normally spend money for durability on items where the technology is rapidly evolving, delivering more for less over short time spans. Few people have cell phones that can last a decade even though they could be built. Indeed, when a component fails, they generally replace the phone for one with upgraded features rather than fixing it. The old phone was “good enough” to serve until dramatically better ones came along.

For some people “good enough” in a cell phone is the latest technology. For others, it’s the “free” phone that’s a generation back but still “good enough” for their purposes which probably don’t revolve around the phone.

“Good enough” shouldn’t be a pejorative. Yes, we can often do things better but usually only by allocating more resources to it without getting any real return on that investment. Indeed, if something else does the same job for less, we normally go for it. We should never stop looking for ways to do more with less – improving efficiency or productivity – but at any given point, we should always make sure that we do is “good enough” for what it is intended to do.

People who say “good enough is never good enough” are just trying to be clever. It’s not how they or anyone living with limited resources actually do things.

Posted in Economics, Environment, Home repair, Internet, Politics, Science and Mathematics | Leave a comment

The Clockwork Mind

“Some believe it is only great power that can hold evil in check, but that is not what I have found. It is the small everyday deeds of ordinary folk that keep the darkness at bay. Small acts of kindness and love. Why Bilbo Baggins? Perhaps because I am afraid, and he gives me courage.”
― J.R.R. Tolkien

There has always been a debate between free will versus destiny. In the last decade it has taken on a new dimension with brain scans revealing that our minds are made up before we are consciously aware of the decision. This has led some people to claim that free will is just an illusion and that we are automatons reacting to our environment but unaware of that fact.

I have to disagree with that view. While I may have subconscious impulses and while many of my actions are done out of habit rather than intention, I can still ponder decisions and change my mind.

Let’s take a concrete example. When I purchased my house, it wasn’t a decision to be taken lightly. I knew what I was looking for in terms of location, price and features. Perhaps my subconscious was churning away but if so, it was doing so because of my conscious decision about what I wanted.

I will also argue that we cannot be mere automatons because that would be impossible. While Sir Isaac Newton contemplated a clockwork universe built around mathematical principles, that notion was shattered when in 1887 mathematicians Heinrich Bruns and Henri Poincaré determined that there is no general analytical solution to the 3 body problem.

This is a special case of what later became known as chaos theory. In complex systems you cannot know the initial conditions accurately enough to predict future states perfectly. Or to use the butterfly effect analogy, the flapping of a butterfly’s wings in Asia can trigger a hurricane half a world away.

When we get into quantum mechanics, things become even clearer. It’s not that we don’t have good enough information but rather that we can’t have good enough information. We cannot precisely know both the position and velocity of an electron at a given moment. Moreover, the very act of measuring either one changes the other.

Einstein always thought that perhaps there were hidden variables that we’d yet to discover to explain the quantum weirdness. He said “God does not play dice with the universe”. However physicist John Bell’s famous theorem states “No physical theory of local hidden variables can ever reproduce all of the predictions of quantum mechanics.” He devised a test which so far, although difficult to run, has come down on the side of “God really does play dice.”

In other words, some things you can’t know until they happen.

So let’s take a look at the nature of good and evil. No one chooses to be either one. However we all make choices continually through our lives that shape who we become. This in turn affects those around us, and we are affected also by the decisions they make. In the end we become what our environment and decisions make us, even if we don’t consciously try to be that.

For example, to quit smoking begins with the decision to do it. Successful quitters look at the various triggers that prompt them to light up then try to avoid or change them. They may still find themselves unconsciously lighting up but if they are determined, they won’t take that as a sign of defeat. They will keep trying.

They will have learned about this by researching how to quit smoking. They will want to quit because they have learned, and care about, the impact it has on themselves and others. They care about the impact it has because…

Our minds aren’t clockwork. We can’t predict our responses and future actions any more than we can accurately predict next month’s weather or the fate of Schrödinger’s Cat. However if we keep making small decisions that move us in the direction we want to go, we may get there.

Or the other end of Bell’s Theorem may prove to be true and the entire universe is entirely deterministic though a mesh of non-local hidden variables that make existence entirely futile. But then I had to write that, didn’t I?


Posted in Education, History, Politics, Religion, Science and Mathematics | Leave a comment

The impact of electoral systems on terrorism

One thing I will note is that nations that use winner-take-all elections are the main targets of these nutbars. When people feel their votes don’t count (a feature of winner-take-all elections), they feel a greater legitimacy to using violence.

Another reason is that some politicians use the attacks to promote their own xenophobic agendas, which makes the nation an even more appealing target to those whose message is “the West hates us”.

A third reason is that these nations generally have poorer social programs than nations using proportional representation. More people are likely to feel left out of the economic life of the nation.

The U.K. and France are the main targets in Europe. The U.K uses first past the post while France uses a two-round system. While terrorism occurs in other European nations, these two seem to bear the brunt. Indeed, one of the exceptions, Norway, spawned a terrorist who claimed to be trying to preserve his nation against immigrants who threatened it.

In the U.K., May is also pushing the Brexit agenda on the backs of refugees. In France, La Pen’s poisonous message was amplified when she made it to the second round of the Presidential election.

And of course I don’t need to remind anyone of what just happened in the U.S. thanks to their double winner-take-all Presidential race. Not only do they elect a powerful President but they do so using a series of winner-take-all state by state races where, save for a few states, the candidate with the most votes in that state gets all of their electoral college votes. The electoral college then hold the actual winner-take-all presidential election.

Electoral systems are far from being the only determinant of a nation’s vulnerability but they do seem, on the surface, to play a role. When you divide the voters into opposing camps the way winner-take-all elections do, anger is almost inevitable.

Posted in Economics, Electoral Reform, History, Politics, Religion, social programs | Leave a comment

when rocks hit people.

I was curling today and ran into an issue of sportsmanship – something that curling is associated with (it’s considered essential). It happened in the first end of a game where I skipped one team but my second throws 4th stones so I was in the house for the final shot.

The shot missed its target but raised another stone into the rings after it deflected off another stone. It didn’t have a lot of momentum but was heading into the 8-foot ring when it was stopped by the opposing vice’s foot.

In this instance the rule is clear that the non-offending team gets to place the rock where they believe it would have ended up had it not been interfered with. It is expected that this will be done in good faith.

Now clearly she didn’t see the rock or she would have avoided it. However she did feel it when it touched her shoe which was half in the 8 foot ring. I did see it and was moving to sweep it but didn’t get the chance.

In my judgment the rock would have moved fully into the 8 foot ring – a distance of 12 – 15″ from the point of impact – but not much further. This led to acrimony from the opposing vice who, despite not seeing the rock when it was moving, believed it would have traveled only a couple of inches.

The difference was significant as they had one stone partially in the 8 foot and another just fully in. They also had a rock midway in the 8 foot. The distance the rock would have traveled would make the difference between them scoring 1, 2 or 3. In fact, if the rock slid 6 inches further than I thought it would, we would have scored 1.

Anyone whose watched rocks slide in the house knows that sometimes rocks slide and sometimes they dig in. It’s not easy to predict. And that’s watching the stone move. We have even less experience feeling stones bump our feet – it just doesn’t happen that often.

As a side point, I will note that the force required to move a stone is actually larger than the force required to stop a stone. That’s because the static friction between two objects is larger than the sliding friction. To move a stone, you have to overcome the static friction. That’s why curlers never start their slide with a stone at rest. However when curlers move rocks with their feet, they are usually at rest so they may mistake the force required to move a stone for the lesser force required to stop one.

I placed the rock where I thought it would have stopped then looked around to see how many points we gave up. It turned out to be 1.

That should have been the end of the discussion. Unfortunately it wasn’t. Because I don’t throw 4th stones, I was in the house and not my vice. The other vice tried to suggest that my vice should have made the call, even though she was not in the house at the time the incident occurred.

The rule actually doesn’t specify that. Anyone on the non-offending team who feels competent in placing the rock can do it. This would be someone who observed the incident. The vices merely agree on the score after each end.

It’s unfortunate that sometimes players do get in the way of rocks and we don’t have instant playback to document the incident. It’s also unfortunate that these incidents typically occur when players are busy in the house and distracted.  However these incidents are generally resolved with no hard feelings and no accusations of malfeasance.

That’s one of the great things about curling.

BTW: the game ended in a tie.

Posted in Sports | Leave a comment