Radar trends to watch: October 2020

This month, the big surprise is that there’s no significant technology news about COVID. And there is more news than ever about legislation and regulation. I suspect that the legal system will be a big driver for technology over the next year. Another trend that doesn’t quite count as technology news but that definitely bears watching is that college enrollment in the US is down. Grad schools are up, 4 year colleges are down slightly; the big hit is in 2 year colleges. COVID is probably the biggest contributing factor, but regardless of the cause, this is an inauspicious trend.

AI and Data

• Beyond Text: Some experiments have trained GPT-3 on both text and images, enabling it to generate images from captions.
  
• The importance of time series: I’m less impressed with using time series to estimate the cost of an auto repair than with the possible applications of time series. Time series has had less attention than NLP, but it’s equally important.
  
• Can BERT learn common sense? Possibly. 50% correct answers to “common sense” problems doesn’t sound impressive, but I wonder how well humans would score.
  
• Training AI not to forget using generative replay techniques: One problem with neural networks is that they frequently “forget” important features in the data. Replay is an idea that is based on how humans remember: we can strengthen memories by recalling significant events.
  
• Microsoft Deep Speed uses new parallelism techniques to train very large models (billions of parameters) with less CPU and GPU power.
  
• Microsoft gets an exclusive license to OpenAI’s GPT-3: The public-facing API will remain, but access to the code is limited to Microsoft. As MIT Tech Review points out, this is a significant departure from OpenAI’s vision of freedom from both government and for-profit money.
  
• Biological signatures (e.g., heartbeat) can be used to detect deep fakes. It’s not news that your pulse can be detected in video. But the process of creating a fake distorts these signatures in ways that are detectable.
  
• Diffbot is building a knowledge graph of the entire web. It’s not a language model (though they may add a natural language query feature); it’s a database of facts in subject-verb-object form. Diffbot represents an older approach to machine learning that, in practice, might work better than language models like GPT-3.
  
• Visual reasoning with machine learning enables AI systems to reason about what people are doing and how they are doing it. This is an important step towards building robots that can function properly as assistants to humans.

Law

• Trump’s attempt to block WeChat has been blocked by the courts; now TikTok is in the same state. (The decision allows users to download TikTok; other restrictions that come into effect in November are still in place.) These blocks are only temporary, so presumably the future of these companies in the US is in the hands of the legal system.
  
• I have seen reports that the EU is interested in the TikTok deal for some unsurprising reasons. The EU is also concerned about social media companies headquartered in another superpower amassing their citizens’ data. Unintended consequences?
  
• Content moderation legislation: Brookings reports that legislation is increasingly focused on protecting aggrieved citizens than protecting innovation, and that legislation about content moderation is under consideration in Europe, Brazil, the US, and many other countries.
  
• There has been a lot of regulation surrounding privacy, but privacy regulation doesn’t address the problems raised by biometrics (including face recognition). A key issue is what happens when data is used outside of its original context: for example, when drivers license photos are used by agencies like ICE. AINow has published a report on the need for biometric regulation.

Virtual and Augmented Reality

• VR/AR glasses again: Facebook projects releasing the “next step on the road to augmented reality glasses” next year. It apparently won’t have a projector. One suspects it will have a camera, privacy concerns notwithstanding.
  
• The virtual trading floor: Investment companies are starting to use VR so traders can experience the trading floor while working from home. (Is this really a good idea?)
  
• Augmented reality for virtual geology probably isn’t a killer app, but it’s a useful tool for a science that can often involve a lot of travel. The authors say that AR leads to a better learning experience than VR because the students can actually see the teacher, rather than an avatar. Developing for holographic user interfaces may be the next step after Virtual Reality.

Cloud

• “Serverless 2.0” from Lightbend: Cloudstate implements function-as-a-service with stateful functions. It remains to be seen how well this will catch on, but until now, FaaS has required functions to be stateless.
  
• Microsoft’s Azure Arc lets you integrate on-premises servers or Kubernetes clusters with an Azure cloud. There’s been a lot of talk about hybrid cloud, but this strikes me as the most seamless integration possible.
  
• Google adds Confidential VMs and Confidential Kubernetes Engines to its cloud. These encrypt data as it’s in use (in memory), as distinct from encrypting data and storage (on disk) or in flight (on a network).
  
• Microsoft is experimenting with holographic storage for the cloud: extremely high volume, long-term storage.
  
• WebAssembly and Cloud Native: Could Wasm become the language for extending service meshes, allowing developers to write in any language they choose?  Google has been pushing this idea for Envoy and Istio.

Quantum Computing

• IBM’s Quantum Roadmap projects a 1000-Qubit machine by 2023. These are “physical” qubits, and may only be the equivalent of 50 or so logical qubits.  (The difference is due to error correction.) Google has published a similar roadmap, with practical quantum computers (1 million physical qubits) by 2029.

Security

• The Grugq’s definition of Cybercraft as a parallel to (and tool of) statecraft needs to inform thinking about security and operations. This is a consequence of a return to geopolitical competition that isn’t limited to the “great powers.”

Programming

• NVIDIA is acquiring ARM for $40B. This acquisition establishes NVIDIA as a powerhouse in general purpose computing, and as serious competition for Intel and AMD in their core businesses. Their intent is clearly to be a one-stop provider for AI and other applications that require high performance arithmetic.
  
• “Everyone will be a software engineer, and barely any will know how to code.”  A quirky but prescient article arguing that the key skill is understanding how to solve problems with software, not produce it. Certainly if no code and low code products democratize software creation, we’ll be left with the problem of understanding what the software should do.
  
• A video transcoder in the browser using Wasm is the most compelling demo of Wasm that I’ve seen yet. Wasm isn’t “coming”; it’s clearly already here. Developers just aren’t using it yet.

Cryptocurrency

• Many countries (the US is notably absent) are building pandemic-relief programs around central bank digital currency (cryptocurrency). This is a stepping stone to broader use of digital currency.