Keith Schacht’s Weblog

Using LLM to process video. I have done a lot of experimenting with passing screenshots into an LLM to give it additional context, but I really want to try passing video directly into the LLM. I believe Gemini is the only one that supports it. This is a summary of Simon Willison’s recent experiment with Gemini for video.