Radar Trends to Watch: October 2023

AI continues to spread. This month, the AI category is limited to developments about AI itself; tools for AI programming are covered in the Programming section.

One of the biggest issues for AI these days is legal. Getty Images is protecting customers who use their generative AI from copyright lawsuits; Microsoft is doing the same for users of their Copilot products.

Also on the legal front: Hashicorp’s switch to a non-open source license has led the OpenTF foundation to build OpenTofu, a fork of Hashicorp’s Terraform product. While it’s too early to say, OpenTofu has quickly gotten some significant adopters.

AI

• OpenAI has announced that ChatGPT will support voice chats. Will its voice persona be as verbose and obsequious as its text persona?
• Getty Image has announced a generative image creation model that has been trained exclusively on images for which Getty owns the copyright. Getty will reimburse customers’ legal costs if they are sued for copyright infringement. Getty is compensating artists for the use of their work.
• Sony and Meta have developed new ways to measure racial bias in computer vision. Sony has developed a two dimensional model for skin tone that accounts for hue in addition to darkness. Meta has released an open source dataset named FACET for testing AI models.
• The Toyota Research Institute has built robots with large behavior models that use techniques from large language models. These robots have proved much more versatile and easier to train than previous robots.
• Open AI has released DALL-E 3, a new image synthesis AI that’s built on top of ChatGPT. It is far better at understanding simple prompts without complex prompt design. It will become a feature of ChatGPT+, and has been integrated into Microsoft’s Bing.
• In an effort to throttle a flood of AI-generated books, Amazon has limited authors to three books per day. That still seems like a lot—it’s unlikely that a human author could produce one book per day, let alone three.
• Updates to Google’s Bard include integration with Maps, Google Docs, and a “Check your answer” button. Checking seems to be limited to verifying facts using search results (for which Bard gives citations), but it’s still useful.
• Optimization by Prompting is a new technique for developing effective prompts. OPRO uses an AI model to optimize the prompts used to solve a problem. Starting with “Take a deep breath” evidently helps.
• Google’s DeepMind has developed an AI model that can identify variants in genes that could potentially cause disease.
• Competition in the vector database space is heating up. LanceDB is yet another entry. It is open source, and is designed to be embedded within apps, with no external server to manage. Data is stored on local hard disks, making it conceptually similar to SQLite.
• Stability AI has released a new demo of generative AI for music, called (unsurprisingly) Stable Audio. Generative AI approaches to music lag behind generative art or text, but Stable Audio has clearly made some progress.
• Microsoft has announced that it will assume liability for copyright infringement by all of its Copilot products (not just GitHub). They claim to have built guardrails and filters into their products to prevent infringement.
• HuggingFace now offers Training Cluster as a Service. This service allows you to use their infrastructure to train large language models at scale. The home page lets you build a cost estimate, based on the model size, the training data size, and the number and type of GPUs.
• Pixel tracking means something different now. MetaAI has announced CoTracker, a Transformer-based tool that tracks the movement of multiple points through a video. Source code is available on GitHub under a Creative Commons license.
• Google has released DuetAI, its AI-driven extensions to its Workspace suit (GMail, Docs, etc.). Although there is a free trial, there will be an additional fee for using Duet. It can take notes on meetings in Google Meet, write emails and reports, participate in chats, and more.
• Google’s DeepMind has launched SynthID, a watermarking tool for AI images. It includes tools for watermarking and detecting the presence of watermarks. SynthID is still experimental, and only available to users of Google’s Imagen, which itself is only available within Vertex AI.

Programming

• The free, open source Godot game engine is proving to be an alternative to Unity. While Unity has (mostly) backed off from its plans to require per-install fees, it has lost trust with much of its development community.
• OpenTofu, OpenTF’s fork of Hashicorp’s Terraform, has been backed by the Linux Foundation and adopted by several major enterprises.
• DSPy is an alternative to Langchain and Llamaindex for programming applications with large language models. It stresses programming, rather than prompting. It minimizes the need for labeling and “prompt engineering,” and claims the ability to optimize training and prompting.
• Zep is yet another framework for building applications with large language models and putting them into production. It incorporates Llamaindex and Langchain.
• Tools that analyze source code and trace its origins in open source projects are appearing. The development and use of these tools is driven by automated code generators that can infringe upon open source licenses.
• The WebAssembly Go Playground is a Go compiler and runtime environment that runs completely in the browser.
• Wasmer is a sandbox for running WebAssembly apps. It allows you to run Wasm applications on the command line or in the cloud with extremely lightweight packaging.
• Guidance is a programming language for controlling large language models.
• Microsoft and Anaconda have launched Python in Excel, which allows Excel users to embed Python within spreadsheets.
• Rivet is a graphical IDE for developing applications for large language models. With minimal coding, users can build prompt flows, using tools like vector databases. It’s part of a growing ecosystem of low-code tools for AI development.
• JetBrains has released RustRover, a new IDE for Rust. RustRover does not incorporate AI, although it does have the ability to suggest bug fixes. It supports collaboration, and integrates GitHub, the Rust toolchain (of course), and unit testing tools.
• Refact is a new language model that is designed to support refactoring; it includes fill-in-the-middle support. It is relatively small (1.6B parameters), and has performance equivalent to other publicly testable language models.
• HuggingFace has developed a new machine learning framework for Rust called Candle. Candle includes GPU support. The GitHub repo links to a number of examples.

Security

• Google, Apple, and Mozilla have reported a severe vulnerability in the WebP image compression library that is actively being exploited. Fixes are in the current stable release of Chrome and other browsers, but other applications that rely on WebP are vulnerable.
• The NSA, FBI, and Cybersecurity and Infrastructure Security Agency have published a CyberSecurity Information Sheet about Deepfakes that includes advice on detecting deepfakes and defending against them.
• Google is releasing an API for their Outline VPN to developers to build the VPN into their products. Outline has been useful for evading government censorship. The API and SDK will make it easier to build workarounds when governments learn how to detect the use of Outline.
• Any sufficiently advanced uninstaller is indistinguishable from malware. You have to read it just for the title. A nice piece of analysis.
• Security breaches frequently occur when an employee leaves a company, but retains access to internal apps or services. Just in time access minimizes the risk by granting access to services only as needed, and for a limited time.
• Few security stories have happy endings. Here’s one that does: the FBI managed to infiltrate the Quakbot botnet, redirect traffic to its own servers, and use Quakbot to automatically uninstall its own software.
• How do you maintain security for software that’s updated from a repository? Proper key management (including keeping keys offline) and expiring old metadata are important.
• MalDoc is a new attack in which a Word document with malicious VB macros is embedded in a PDF document. The document is treated as a PDF by malware scanners, but can be opened either as a Word document (which executes the macros) or as a PDF.

Privacy

• Research by Mozilla has shown that connected cars are terrible for privacy. They collect personal data, including video, and send it back to the manufacturer, who can sell it, give it to law enforcement, or use it in other ways without consent. Management of the data doesn’t meet minimum security standards.
• The Signal Protocol, a protocol for end-to-end encryption, has been upgraded for post-quantum cryptography. The Signal protocol is used by the Signal app, Google’s RCS messaging, and WhatsApp.

Web

• Two new decentralized projects provide services that previously were only available through centralized servers: Quiet, a team chat app that’s an alternative to Slack and Discord; and Postmarks, a social bookmarking service that’s a successor to the defunct del.icio.us.
• Wavacity is the Audacity audio editor ported to the browser: another tour de force for WASM.
• Cory Doctorow’s interview about saving the open Web is a must-read. Interoperability is the key.
• Web LLM now supports LLaMA 2 in the browser! Everything runs in the browser, using WebGPU for GPU acceleration. (Chrome only. Be prepared for a long download when you try the demo.)

Hardware

• Humanity’s oldest writing is preserved on ceramics. That may be the future of data storage, too: a startup has developed ceramic-coated tape with storage of up to 1 Petabyte per tape. A data center could easily house a Yottabyte’s worth of tapes.
• Qualcomm is making a big investment in RISC-V. RISC-V is an open source instruction set architecture. We’ve said several times that RISC-V is on the verge of competing with ARM and Intel; adoption by a vendor like Qualcomm is an important step on that path.

Quantum Computing

• Researchers used a quantum computer to slow down a chemical process by a factor of 100 billion, allowing them to observe it. This experiment demonstrates the use of a quantum computer as a research tool, aside from its ability to compute.
• IBM has announced a significant breakthrough in quantum error correction. While QEC remains a difficult and unsolved problem, their work reduces the number of physical qubits needed to construct a virtual error-corrected qubit by a factor of 10.

Biology

• DIY tools that automate insulin delivery systems for managing diabetes are becoming accepted more widely, and can significantly outperform commercial systems. One DIY system has received FDA clearance.