allow patching selectively window.ai even when window.ai natively available

[xmlking]

lets say browser natively support window.ai.writer , window.ai.rewriter , window.ai.summarizer etc because either user enabled them via chrome://flags/ or website added origin-trials for them, today aibrow only polyfill all features or nothing.

It would be nice if developers or users can be able to choose selective models from aibrow and others from native chrome AI models. for example I like to use only window.ai.writer , window.ai.rewriter polyfilled by aibrow and rest from browser native models.

Ask:

1. Allow users to choose nativeAI vs aibrow for writer, rewriter, summarizer, languageModel, languageDetector and translator usecases individually via Extensions settings page
2. Allow developers choose nativeAI vs aibrow for writer, rewriter, summarizer, languageModel, languageDetector and translator usecases individually via API

[Thomas101]

In terms of individually choosing the APIs, this is a transient thing until Chrome takes them out of the origin trial. I'm hoping that they'll reach some level of stability soon, but I think this is something to come back and look at depending on what the final set of APIs looks like. What I have added, is a setting in the extensions settings to always override window.ai with window.aibrow. So if, for example, a user wants to use our model over the built-in browser, there's an option to do this.

There should already be enough stuff exposed from a developer standpoint to pick and choose which API to use.

• window.ai - poly-filled by aibrow if not available or overwritten if the user has explicitly elected to overwrite it
• window.ai.aibrow === true - can be used to detect if aibrow is poly-filling the API
• window.aibrow - always added as a variable
• window.aibrow.browserAI - a reference back to the original window.ai, even if overwritten

This means a developer should be able to do something like this to pick and choose which API's they want to use:

function getBrowserAI () {
  // Always get the browsers AI, regardless of the user's preference in AiBrow
  const ai = window.ai && window.ai.aibrow === true
    ? window.ai.aibrow.browserAI
    : window.ai
  if (!window.ai) {
    // Eventually this check wont be needed as all browsers support window.ai
    throw new Error('Your browser doesn\'t support window.ai')
  }
  return ai
}
function getAiBrow () {
  if (!window.aibrow) {
    // Send user to the download page
    throw new Error('AiBrow is not installed')
  }
  return window.aibrow
}

const browserAI = getBrowserAI()
const aibrowAI = getAiBrow()

// Both can be then used interchangeably as needed
await browserAI.summarizer.create()
await aibrowAI.summarizer.create()

// Or you can fallback as needed
const writer = browserAI.writer || aibrowAI.writer || throw new Error('Writer is not supported')
await writer.create()

[xmlking]

Thanks for sharing ideas how to pick and choose specific feature from each AI implementation. I will try this approach.

What I have added, is a setting in the extensions settings to always override window.ai with window.aibrow. So if, for example, a user wants to use our model over the built-in browser, there's an option to do this.

Is this override feature part if 1.2.0 or 1.2.1 ? I cannot find in 1.2.0

I was trying yesterday, to override globally within app, specific features but I understand they are read-only and can I cannot override :)

https://github.com/xmlking/spectacular/blob/3aa463f5c14afc1ca31f9a99f3463893ea5ac59b/packages/smart/src/components/smart.ts#L37

[Thomas101]

It's 1.2.1. We submitted it to the Chrome store yesterday but are still awaiting approval and publish :)

[rhys101]

Version 1.2.1 is now published live on the Chrome store.

[xmlking]

My chrome version : 131.0.6778.86
AiBrow Versions:
Extension: 1.2.1
On-device helper: 1.2.1

Q: if I enable Override browser provided window.ai in settings, should I see print statement for below code below?

[xmlking]

apparently switching Override browser provided window.ai is not consistent
It may work first time, but not on repeated switching

[Thomas101]

I'm guessing from the logs you were toggling it on and off and then trying the setting on the page. After changing the setting, you'll need to reload any tabs where you want to test it because the content-scripts and runtime's already setup with the setting that were available on navigation

[xmlking]

I'm guessing from the logs you were toggling it on and off and then trying the setting on the page. After changing the setting, you'll need to reload any tabs where you want to test it because the content-scripts and runtime's already setup with the setting that were available on navigation

reload some times helps to reflect changes I made in setting, but when see contentscript-main-override.js errors, that is when I see inconsistency issues. When it happens window.aibrow also undefined

[xmlking]

seams like, when try the tests in chrome console where I open https://demo.aibrow.ai/playground/, the switching behaviour is working correctly.

previously I was testing on google.com home page console, I see aibrow not found logs from content.

once its works, it works as expected in any website console.

[xmlking]

Other behaviour I noted is, when I have origin-trial in my home page, Polyfilling is not happening when I toggle Override browser provided window.ai check box
e.g.
https://github.com/xmlking/spectacular/blob/main/apps/console/src/app.html#L10-L20

[Thomas101]

Thanks, I've tightened this up so it should be more reliable. It will go out in a release next week - version 1.2.2

[xmlking]

b.t.w @types/dom-chromium-ai 0.0.3 released

DefinitelyTyped/DefinitelyTyped#70757