Up next in 10
Article on Cult of Mac: https://www.cultofmac.com/how-to/apple-intelligence-ios-18-2
More Apple news: http://www.cultofmac.com
Produced by Extra Ordinary for Cult of Mac
Music composed by Will Davenport, arranged by D. Griffin Jones
Chapters:
0:00 Intro
0:27 Image Playground
3:32 Image Wand
5:14 Genmoji
6:55 Visual Intelligence
9:25 ChatGPT in Siri
10:34 Writing Tools
11:53 New Siri APIs
Follow us!
Threads: https://www.threads.net/@cultofmac
Mastodon: https://mastodon.social/@cultofmac
Instagram: https://instagram.com/cultofmac/
Show More Show Less View Video Transcript
0:00
iOS 18.1 rolled out the first set of
0:02
Apple intelligence features mostly
0:04
setting the table with the basics like
0:06
summarization writing tools and image
0:08
cleanup this second round of features an
0:10
iOS 18.2 expected to be released in
0:13
December rolls out the most hotly
0:15
anticipated features image playground
0:18
gen Emoji visual intelligence chat GPT
0:22
integration and so much more let me show
0:24
you how it
0:26
works let's start with image playground
0:29
which lets you generate AI images from
0:31
scratch you'll find this app on your
0:33
home screen after you update to 18.2 you
0:35
can just tap the plus button on the
0:37
bottom to generate a new image you can
0:40
also access this in messages inside the
0:43
app menu image playground gives you four
0:45
ways to generate an image you can tap
0:47
describe an image and then type in
0:49
whatever you want and image playground
0:51
will generate one based on your
0:53
description tap the choose person button
0:56
in the bottom right to pick a photo of
0:58
somebody you know this draw from the
1:00
people and pets section of the photos
1:02
app so make sure you have all the faces
1:04
tagged with names for this feature to
1:06
work best you can also tap appearance
1:09
and tap edit to generate a new person
1:12
from scratch you can also tap the plus
1:14
icon in the very bottom right then tap
1:16
choose photo to pick any picture out of
1:18
your photo library to use as a starting
1:20
point here you can also switch between
1:23
two different visual Styles animation
1:26
and illustration finally you can tap any
1:28
of the suggestion bubbles
1:30
at the bottom of the screen after you
1:32
have a starting image image playground
1:34
will instantly start generating options
1:36
four at a time and you can keep swiping
1:38
over to keep making it generate more you
1:41
can tap on any of Apple's tags on the
1:43
bottom of the screen to add additional
1:45
elements to your image or you can type
1:47
in something using the text box tap on
1:50
any of these floating bubbles to remove
1:52
them from the image when you're finished
1:54
tap done in the upper right to save the
1:57
image you can share it you can delete it
1:59
or you can go back and edit it at any
2:02
point in the future and because this is
2:04
still a beta feature you can tap the
2:06
thumbs up or thumbs down to send
2:09
feedback to Apple personally I'm not
2:11
impressed by image playground the images
2:14
that it makes are frankly terrible
2:17
compared to the state-of-the-art with
2:18
mid Journey it's clearly years behind
2:21
the images are really low resolution it
2:24
struggles to draw a lot of straight
2:26
lines some parts of an image weirdly
2:29
blend together and other parts are
2:32
duplicated it's a neat party trick that
2:34
it can all run on device so there are no
2:36
limits to how many images you can create
2:39
but free unlimited garbage isn't an
2:42
enticing offer also Apple needs to add
2:45
more safeguards to this feature the fact
2:48
that you can generate an image based on
2:50
anything in your photo library and by
2:52
extension anything you can find on the
2:54
internet means you don't need to learn
2:56
any complicated prompt injection it's
2:58
all too easy to generate an image of a
3:02
historical German fascist dictator
3:05
standing on the front lawn of the White
3:08
House or if I were a mean-spirited
3:10
middle schooler I could use it to
3:12
generate embarrassing images of a
3:14
classmate for bullying on harassment I
3:16
don't think Apple should ship image
3:18
playground because it's going to be a
3:20
NeverEnding Battle of stamping out fires
3:23
as people figure out how to use it in
3:25
unseemly ways and for very little
3:27
benefit even the best images
3:30
just don't look that good moving right
3:32
along a similar but related feature is
3:34
called image wand this is basically
3:36
image playground but integrated into
3:38
apps like notes Pages keynote and more
3:42
to use image wand in notes you tap the
3:44
paperclip icon and tap create image
3:47
it'll use the context of the notes
3:49
around it to generate it and just like
3:52
image playground you can add additional
3:54
prompts by typing in the text box remove
3:56
them by tapping on the floating bubbles
3:58
tap done to save it or cancel to delete
4:01
it if you have a rough drawing or sketch
4:03
in your note already you can convert it
4:06
into an AI image tap the pen icon on the
4:09
bottom then tap the image wand tool
4:12
Circle the part of the image you want to
4:13
use as input and then give it a
4:15
description of what you want to see
4:18
image wand uses a new style by default
4:20
called sketch but you can convert to
4:23
animation or illustration by tapping the
4:25
plus icon in the bottom right it seems
4:27
to take a broad sort of inspiration
4:29
ation from your sketch but it puts much
4:32
more weight into your text prompt and
4:35
you can test this for yourself by typing
4:37
in something completely unrelated to
4:39
your drawing and it'll just make that
4:43
instead which kind of defeats the
4:45
purpose in my opinion I can see some use
4:48
for this on the iPhone where at best all
4:50
you can really do is a rough sort of
4:53
sketch with your finger as long as
4:55
you're okay with the ugly generative art
4:56
that it makes that is but if I had an
4:59
iPad with an apple pencil I don't think
5:01
I see much use for it image wand doesn't
5:04
enhance your drawing so much as it
5:06
paints over it so it doesn't matter how
5:08
much detail you add it'll just ignore it
5:11
now let's talk about something that's
5:12
actually pretty cool if you've ever been
5:14
irritated that there's not the perfect
5:16
Emoji for the conversation you're having
5:18
you can create your own using gen Emoji
5:21
to use it switch over to the emoji
5:23
keyboard and tap the New Gen emoji
5:25
button on the right it's also very
5:28
cleverly built into the search field if
5:30
you type something in and find out there
5:32
isn't an emoji for it already tap the
5:34
create new emoji button right there to
5:37
jump in then you can type in a short few
5:39
words describing what you want like
5:41
ostrich or tablet or tissue box you
5:44
don't need to jam packet with a long
5:46
complicated prompt it's already trained
5:49
on the visual style of Apple emoji in my
5:52
testing it's really great at generating
5:54
Emoji for specific dog breeds if your
5:57
dog doesn't look like any of the
5:59
existing Emoji well you can create a
6:01
Bernie's mountain dog or a greyhound or
6:04
a pitbull the same goes for cats too you
6:06
can create a tuxedo cat or a calico cat
6:09
or a Main and the other animals
6:12
that are underrepresented in regular
6:14
Emoji like lizards Birds snakes and
6:18
hamsters it also generates objects as
6:21
well and these have mixed results it
6:23
either looks absolutely perfect like
6:26
exactly what an apple shovel Emoji would
6:28
look like or I also tried creating a
6:31
trombone and it created these horrible
6:35
Abominations and in one instance it just
6:38
straight up made a violin instead it
6:40
also works with people too you can type
6:42
in a name from your photo library and
6:44
you have a few Expressions you can give
6:46
them like smiling laughing
6:49
frowning these look okay but I'll take
6:52
it over image playground any day a new
6:55
feature called visual intelligence is a
6:57
way to look things up or get information
6:59
on things you see out in the real world
7:02
it's available exclusively on the iPhone
7:05
16 click and hold on the camera control
7:07
to open it up and click it again or tap
7:10
one of the onscreen buttons to look
7:12
something up ask will send the picture
7:14
to chat GPT open AI chat bot might be
7:17
able to explain what you're looking at
7:19
and you can even type in follow-up
7:21
questions for more information I pointed
7:23
it at a few odd things sitting around my
7:26
office and I was genuinely impressed by
7:28
what it got right
7:30
but of course I caught a few mistakes
7:32
bear in mind that you can't entirely
7:34
trust chat GPT as a sole source of
7:36
information you should always double
7:38
check the facts when it's something
7:40
important search uses a Google reverse
7:43
image search to try and identify the
7:45
object which can come in handy if you
7:47
want to figure out where you can buy a
7:48
product online visual intelligence has
7:51
other features that are a little more
7:52
context dependent if you point the
7:54
camera at a document or a poster that
7:56
has event information on it you can
7:59
quickly add that event to your calendar
8:01
if it's a music festival or concert or
8:04
something that's known on the internet
8:06
it might add additional details in as
8:08
well my testing for this has been a
8:10
little spotty but it could be super
8:13
useful down the road if you take a
8:15
picture of a building visual
8:16
intelligence will reference your
8:18
location with Apple Maps data to look up
8:21
the restaurant or business you're
8:22
looking at you can instantly see hours
8:25
reviews a website bring up a menu and
8:28
more if you're to decide where to eat
8:31
this could be a really handy way to
8:32
quickly get information rather than
8:35
manually typing in every name you see
8:37
technically speaking none of these
8:38
features are super smart chat GPT and
8:41
Google reverse image search are both
8:43
thirdparty Services playing information
8:45
out of the camera like events phone
8:47
numbers websites is based on live text a
8:50
feature the iPhone has had for years and
8:53
visual intelligence doesn't actually use
8:55
a lot of AI to identify a business it's
8:58
mostly just using Apple Maps GPS and
9:01
Compass information which any iPhone is
9:04
capable of now in practice these are
9:06
still practical features I just think
9:08
that without Apple Intelligence being a
9:10
big marketing and branding push Apple
9:13
would be a little more upfront about
9:14
this being a convenient repackaging of
9:17
existing features rather than a new AI
9:20
technology of course there's more chat
9:22
GPT in iOS 18.2 it's now seamlessly
9:26
integrated into Siri if you ask Syria
9:29
broad World Knowledge question instead
9:32
of showing you Google search results
9:34
It'll ask you if you want to ask chat
9:36
GPT tap ask and it'll pass your question
9:39
along you can also ask Siri questions
9:42
about what you're looking at if you ask
9:44
what's this while you're looking at a
9:46
photo reading an article browsing the
9:48
web or watching a video it'll take a
9:50
screenshot and ask for your permission
9:52
to send it along to chat GPT it'll
9:55
analyze it and give you back an
9:57
explanation Apple will remind you that
10:00
mistakes can occur verify details but it
10:03
could provide at least an introductory
10:05
explanation if you don't want to be
10:06
asked permission every time you can turn
10:08
this off in settings Apple intelligence
10:10
and Siri chat GPT by disabling confirm
10:14
chat GPT requests anyone will be able to
10:17
use chat GPT for free with no account
10:20
with a limited amount of daily usage if
10:23
you have a chat GPT account and you want
10:24
greater access you can tap sign in you
10:27
can also upgrade to chat GPT plus
10:30
directly from this page for $1 19999 a
10:33
month the system writing tools for proof
10:35
reading rewriting and formatting text in
10:38
iOS 18.1 have two additional features in
10:41
18.2 the existing writing tools are all
10:44
about pref freting or editing existing
10:46
text but there's a new compose button at
10:49
the bottom that lets you create text
10:51
from scratch based on a prompt powered
10:54
by chant GPT 40 there's a drop- down
10:57
menu where you can select how much
10:58
context give it whether you want to
11:00
share the existing document as part of
11:02
the prompt you can also tap the Plus to
11:04
upload a photo or a document as an
11:07
additional prompt composing with chat
11:09
GPT might take a moment because this is
11:11
a thirdparty service that can't run
11:13
locally on your device but chat GPT is
11:16
widely considered to be the Best in
11:18
Class for writing text you'll see a text
11:20
box where you can describe your change
11:23
if you want something unique you can ask
11:25
writing tools to make significant
11:27
changes to a block of text if in some
11:29
weird and wonderful ways you can try
11:32
things like make it sound Shakespearean
11:35
or make everything opposite which
11:38
usually has pretty hilarious results you
11:41
can also ask for very specific
11:43
directions like make every verb
11:46
italicized or make everything title case
11:49
both of which genuinely impressed me
11:51
with their accuracy finally this latest
11:54
beta includes the First new Siri and
11:56
apple intelligence features available to
11:58
thirdparty app apps Siri will be able to
12:01
see the contents of your screen so you
12:03
might be able to ask it context
12:05
sensitive questions like what should i
12:07
title this presentation what else can I
12:09
add to this email or what's this website
12:12
I'm looking at this is an important step
12:14
towards the new Siri but it's mostly a
12:16
behind the-scenes change it's highly
12:18
dependent on whether apps are updated to
12:21
use it in the first place and at this
12:23
stage in the beta testing process not
12:26
even all of Apple's own apps have been
12:28
updated yet
12:29
but that is the second round of Apple
12:31
intelligence features remember to like
12:33
And subscribe for more updates I'm dein
12:36
Jones with Cult of Mac
#Mobile Apps & Add-Ons
#Clip Art & Animated GIFs
#Intelligent Personal Assistants