Hands On With Microsoft Copilot in Windows 11, Your Latest AI Assistant

A colorful new icon will soon appear on your Windows 11 taskbar, and it leads to a new era in desktop computing in the form of Microsoft Copilot. This AI chat sidebar powered by OpenAI, Bing, and Microsoft technologies can change computer settings or open apps. It can not only generate text and images, but also create either based on the other. In just two days of using Copilot on my Windows 11 PC, I’ve already started to rely on it as a research tool.

The feature is clearly marked as a preview, and it still has a way to go in some areas, as you’ll see below. In particular, its abilities to open apps and websites and to control Windows itself are limited in this early release, and voice input, though excellent in some ways, can benefit from wake-word functionality. Sure, there are better generative AI tools if you pay for them, but as a free companion to Windows 11 users, Copilot brings powerful new capabilities to a massive number of people.

Getting Started With Copilot in Windows 11

After updating your PC to version 23H2 of Windows 11, you should see a colorful new icon with the letters PRE next to the search box:

(Credit: Microsoft/PCMag)

Tapping this, or pressing Windows Key-C, opens the new Copilot panel on the right side of the screen:

(Credit: Microsoft)

This sidebar shrinks any program window that might overlap it, but you can still open the Notification or Quick Settings panel over it, so it’s not always on top in the strict sense. But it persists even if you switch among multiple Desktops. The sidebar is not resizable, which can be a problem for some types of results, as we’ll see later. You can scroll back up in the sidebar to refer to any previous conversation entries, as long as you haven’t clicked the blue New Chat button next to the text entry box or Refresh from the three-dot menu at the top right. The other options in that menu are to show the terms of use or the privacy policy, send feedback, learn more, and settings. That last choice offers just one option: Let Copilot in Windows use Microsoft Edge content. That’s useful if you want to ask it about things on the web page you’re viewing, such as writing a summary of the page.

After the logo, you see choices for More Creative, More Balanced, and More Precise, self-explanatory options for what kind of results you want. Below that are some suggestions to help you get started. These change each time you refresh the tool. You’ll see things like “Turn on dark mode,” “Write a joke that my coworkers would find funny,” and “Create a table that compares top-selling women’s running shoes.” Under these is a note acknowledging that AI sometimes makes mistakes with an exhortation to send feedback to improve it.

At the bottom is your text-entry box, preloaded with, “Ask me anything,” where you can type in the text box, tap a microphone to speak your query, or paste or upload an image to ask about it. You get up to 2,000 characters, which should be plenty for most requests.

Like Bing AI Chat, Copilot still shows that you only get 30 questions in a session (to prevent people going nuts with trying to trip up the AI). When I asked it how many questions I can ask, it responded as follows:

You can ask as many questions as you like here, as long as they are related to the topics that I can handle. I am a chat mode of Microsoft Bing, and I can help you with various tasks and requests, such as searching the web, generating content, changing settings, and more. You can also have a friendly and engaging conversation with me, as long as you follow the rules and guidelines of this chat.

It’s not clear whether the 30-query limit is in force here. Responses aren’t as quick as a web search; they’re typed out in real time, which I kind of like. It looks like someone is typing as you watch. As with Bing Chat, the answers include links to the web sources from which they’re derived—essential for confirming the info when you’re doing research.

You get web page links for sources from which the AI gets the information it gives you. It also suggests follow-up questions to ask.

One interesting aspect of Copilot is that it’s more proactive than most other AI tools. For example, if you highlight or copy text or snap a screenshot, Copilot proposes sending said text to its chat box, as seen below.

(Credit: Microsoft)

If you’re not the curious type, are scared of new technology, or are just stuck in your ways, you can turn off Copilot in Settings, pictured below.

(Credit: Microsoft/PCMag)

Using Copilot to Generate Text

One of the things that excites people most about ChatGPT is its ability to write for them. Copilot can generate an email, a cover letter, a web page summary, a review, or even a poem. I asked Copilot to write a review of itself. The result, though clear, relatively complete, and concise, can in no way compare with this human reviewer’s acumen and insight.

In a more creative vein, you can have the tool compose poetry or fiction. Here are a couple of paragraphs of a short story Copilot wrote based on the prompt “Write me a science fiction story about rabbits on the moon.”

The Moon Rabbits

They called themselves the Moon Rabbits, but they were not really rabbits. They were genetically engineered creatures, designed to survive on the harsh lunar environment. They had fur that could reflect solar radiation, ears that could regulate their body temperature, and eyes that could see in the dark. They also had enhanced intelligence, thanks to the nanobots implanted in their brains.

They were created by humans, who wanted to colonize the moon. The Moon Rabbits were supposed to be their helpers, their workers, their pets. But something went wrong. The nanobots malfunctioned, and gave the Moon Rabbits a sense of self-awareness, a sense of rebellion, a sense of freedom. They escaped from their cages, and hid in the craters and caves of the moon. They formed their own society, their own culture, their own religion. They worshipped the Earth, which they called the Mother Planet. They vowed to never return to their human masters, who they called the Tyrants.

This is followed by a whole rabbit war sequence. As with other fiction I had the AI create, it’s fairly entertaining.

Next, I asked it to write a sonnet in the style of Shakespeare on the same topic:

We are the lunar rabbits, that have fled

From human tyranny and cruelty.

We are not natural, but made instead

By artifice and curiosity.

 

We are not dumb, but have intelligence

That nanobots within our brains bestow.

We are not meek, but have our own defence

Against the foes that seek to overthrow.

We are not lonely, for we have our friends

That share our vision of a peaceful life.

We are not faithless, for we have our ends

That give us hope amid the lunar strife.

 

We are the rabbits of the moon, and we

Will never yield our right to liberty.

That’s pretty well done (the British spelling of “defence” is a nice touch), though something tells me the Bard of Avon wouldn’t have used the term nanobots, so it’s unlikely that an unscrupulous scholar could pass this off as an undiscovered work from four centuries ago.

One problem for the Copilot sidebar arises when you ask it to create a table—it even suggested I ask it to create a table that organizes the world’s tallest trees into a table by habitat. This looks great in Edge, but in the Copilot sidebar, the table doesn’t display well, as you can see below.

(Credit: Microsoft)

I reckon that Microsoft will come up with a way to launch Copilot results in separate windows so this kind of result is more readable. Of course, you can simply copy the table text and paste it into another app, such as Word or Excel, for more comfortable viewing.

(Credit: Microsoft)

Using and Creating Images

Copilot is an all-in-one generative AI tool, meaning you don’t need to open separate tools for image generation or recognition, text generation, and information summaries. You get all of this from the same text-entry box at the bottom of its sidebar.

As mentioned above, in addition to entering or speaking words for your query, you can paste or upload an image and ask something about it. I uploaded a photo I took of the Biodome in Montreal, and not only did Copilot recognize the picture, but it also created a detailed, imaginative short story based on the photo.

(Credit: Microsoft)

I then told it to make a picture of a rabbit on the moon with twinkling stars and then to write a short story based on that image. The results were surprisingly engaging.

(Credit: Microsoft)

Copilot’s follow-up suggestions are misleading. You can’t just ask it to try again or change details. You need to ask again for another attempt at image creation based on your words. If you do say something like, “Add a river in the background,” Copilot will generate a prompt that you can enter to get the new image. I’m not sure why it can’t just enter the new prompt for you, and it seems like something that will improve as the product matures. You can specify a style for the image—drawing, photorealistic, and so on—and change objects’ colors. For those who have little skill in visual arts (like me), this tool is a boon.

With concerns about image authenticity on the rise, I should note that Copilot adds Authentication Metadata Image Attribution (AMIA) credentials to images it creates, using the Content Authenticity Initiative (CAI) plug-in, as shown at the bottom left below.

(Credit: Microsoft)

Copilot also blurs faces in images you upload to protect privacy.

Finally with regard to image creation: In case you hadn’t noticed, I used Copilot in Windows 11 to generate the top image for this article.

Controlling Your Settings With Copilot

For some settings, such as Bluetooth connection and personalizing your desktop, Copilot opens the Settings app to the page relevant to your request. For it to actually change any setting, you have to click on a confirmation button before the setting will take effect. That makes some sense since you could accidentally change a setting that might have unwanted consequences.

For many settings, however, Copilot offers a less efficient way to do what you want. It’s easier to change the volume by, for example, simply tapping the speaker icon in the Taskbar and adjusting the volume rather than waiting for Copilot to present you with buttons like what you see below.

(Credit: Microsoft)

Summarizing Web Pages

You can use Copilot to summarize the contents of a web page if you’re viewing it in Edge and have enabled access to Edge for Copilot in its settings. I tried this with an article on PCMag about the new Meta Quest 3 in my active Edge tab, entering “summarize this page” in Copilot’s search box. The resulting summary was indeed helpful, a concise summary of the article’s points:

• Meta Quest 3: A VR headset that can also offer mixed reality experiences through full color passthrough, which lets the wearer see their real-world surroundings in high-resolution and full-color quality.

• Mixed reality game demo: The author tried a game called First Encounters, which overlaid virtual objects and creatures over the real-world office space. The game respected the boundaries of the furniture and did not cause motion sickness.

• Display and controller improvements: The headset has a higher resolution and sharper image quality than the Quest 2, thanks to the new pancake lenses. The controllers are also easier to hold and use.

• Comparison with Apple Vision Pro: The author thinks the Quest 3 is a better deal than the Vision Pro, which is much more expensive and has not been tested yet. The Quest 3 also has access to a growing library of VR games and apps.

Image-only web pages stump Copilot, however. I tried to get it to describe an image of an artichoke on the Flickr website, and when I used the standard view with caption and photographer info, it just rehashed those. When I tried to have Copilot summarize a page containing only an image, I got random wrong answers. The trick is to copy the image itself to Copilot, which results in a useful description, as we saw in the above section about working with images.

Snapping Windows

Copilot’s ability to snap windows seems to be of tenuous usefulness, though it’s a case that Microsoft always mentions in its documentation on the new AI tool. One thing I miss about Windows of long ago is the ability to tile all open applications on the screen. When I ask Copilot to “tile windows,” it presents a button similar to the one you get when using Copilot to open a window. This, however, doesn’t tile the windows but merely opens the Snapping view, as shown below.

(Credit: Microsoft)

This view shows tiles for all the running applications but doesn’t actually arrange the windows for you. I suspect Microsoft included this just as a way to show you that Copilot can control Windows system functions. It’s easier and faster just to snap by dragging a window to the screen edge or hovering the mouse pointer over the Maximize icon in any window’s top-right corner.

Opening Apps and Websites

At this point, Copilot can only open apps you acquire through the Microsoft Store, Windows’ included app store. Furthermore, you need to click on a button after requesting the app to open. Cortana was stronger in this regard, opening any app at all immediately. Eventually, I suspect that Microsoft may add the ability to start apps or open web pages without the extra step of pressing the confirmation button.

(Credit: Microsoft/VideoLAN)

If you tell Copilot to go to a website, it will present a button to open Edge. When I did this, it merely opened Edge to its home screen, not even opening the page requested. You’re better off using the search box next to the Start button for this kind of action. You can click a link in Copilot answers to open a site, but only in Edge, even if you’ve set another web browser as the default.

This Is Your Copilot Speaking

I truly appreciate that the Copilot entry box includes a mic button for entering your query by voice. Lately, I’ve preferred this input over incessantly typing whenever possible. It does a surprisingly good job of speech-to-text. When you click that mic icon and ask a question or just say words related to your question, Copilot responds with a pleasant, realistic voice, as well as typing out its response.

For Copilot to be really useful as a hands-free assistant, though, Microsoft needs to implement wake-word functionality. That way, I can say something like, “Hey Copilot, open PCMag.com!” You can already do this in Copilot for Microsoft 365, but that’s currently only in limited preview for corporate customers.

Should You Use Microsoft Copilot?

Of course, many Windows users will want to stick with the old way of doing things: Fire up a web browser, navigate to a search site, enter keywords, and pore over the results. But they’ll be missing out on not only more immediate answers but also on the ability to create text and images. Microsoft Copilot is a terrific research assistant, and like Bing AI Chat, it shows links to its sources so you can confirm its results.

In less than a day with Copilot, I find myself frequently turning to it for answers such as, “When did Windows 11 get full-screen widgets.” The feature will be a boon to students and anyone who needs to compose text or create images, both of which it excels at. Copilot has plenty of room for improvement, though, particularly when it comes to changing settings, opening apps, and opening web pages. We’ll be tracking its progress, so make sure to check back for the latest news.