Alexito's World

A world of coding đŸ’», by Alejandro Martinez

WWDC 18 session notes

As every year after WWDC is time to sit down and watch the videos of the sessions! I wrote last year about How I watch the WWDC videos and this year I'm doing a similar thing. I still use the unofficial WWDC for macOS to download the videos and watch them at more than 2x in VLC. This time I decided to use a single note in Notes app to keep my notes of the sessions and to mark the videos I've seen. And because everything is in a single note it makes it really easy to share, so I decided to share my notes with you!

This is a copy of the raw notes that I keep, so it may contain some mistakes or typos. It's also not a full fledged summary of the talks but instead it reflects my interests on the topics and what I thought would be interesting to have as a quick reference. Some talks may not have notes because I was not interested on the topic (or because I watched them at midnight 😅).

2018-06-10: I'm sharing them without having watched all the sessions yet. 101 videos are not watched that quick :P

What's New in Swift

Everything was known thanks to being Open Source! 🎉

Creating Apps for a Global Audience

Font cascade lost for when a font doesn’t support a language script.

Use proper api to scale fonts based on dynamic sizes

Localised emphasis should be done with bold or color changes in words instead of cursive as not all languages have cursives.

Introducing Podcast Analytics

iTunes U content in Podcast app.

Modern Podcast tags: Trailer, Season, Bonus, Episode Number

Podcasts for Apple Watch.

Chapter support for mp3 including chapter art.

Group podcast notifications.

Anonymous analytics.

What's New in Cocoa Touch

DataSource prefetching is useful to avoid lost frames on cellForRow but iOS 11 could be asking for prefetch at same time was requesting a cell, iOS 12 is smarted and doesn’t do that so the CPU prioritizes displaying the cell

iOS 12 passes UIKit scrolling info to the CPU so it can rump up faster and on time to avoid dropping frames.

Memory allocation can have impact on performance, if big chinks are asked and there is no free memory system will give it from other apps. First, users will see other apps get closer when they don’t expect it, but your app will have a performance it while the system goes and reclaims that memory. so yeah, gg for random allocators

iOS 12 will configure image backing stores automatically to reduce the memory usage, you can configure it manually if you know what you want (grayscale vs deep color)

Autolayout internal changes, faster by default.

UIKit swiftyfied!

Name-spacing and nesting

free functions as methods

Rect, size, edges and other types now conform to Codable!

Notifications actions can now be dynamic and changed at runtime

Notifications can have live interactive views

Grouping respects thread identifier

Deep-link into your app notifications settings for granular controls

Apps can save password in login screens into keychain!

Password generation can be customized to follow your app requirements

Siri shortcut following NSUserActivity

Custom Siri intents

Or let users create a shortcut!

Using Accelerate and simd

Live Screen Broadcast with ReplayKit

Can check for is being recorded hide sensitive information

I Have This Idea For An App...

Tutorial for making an app ¯_(ツ)_/¯

Automatic Strong Passwords and Security Code AutoFill

Autofill:

associated domain, done with universal links

Username content type

Password content type

WKWebview login autofill

Save credentials after login:

Automatic, based on content type, fields being removed from the hierarchy and cleared

For suggesting strong password user newPassword content type

Custom password rules in the textfield

Check with custom password validation tool website

tag one time code for sms codes

federated authentication, SSO

ASWebAuthenticationSession, shares cookies with Safari

Measuring and Optimizing HLS Performance

Streaming stuff ¯_(ツ)_/¯

What’s New in ARKit 2

seems easy and cool

the demo of playing a video when detecting a picture is nice

Best Practices and What’s New with In-App Purchases

Subscriptions Introductory discounts

Free trials for non-subscriptions apps, kind of weirdly faked with multiple in-app purchases

StoreReviewController is limited but doesn’t avoid asking to often itself until it reaches the limits,

so you want to keep track of every requests, rate-limited yourself and don’t ask in the wrong moments.

app store URL link maker now can add an action to go to the review page directly

sandbox environment, compresses time of subscriptions 1 year - 1 hour, can request expired subscriptions or even simulate when a kid asks to buy to a parents phone.

sandbox account in the phone is different now! you don’t need to logout from your real account!

receipt has a lot of info about in-app purchases and original purchases from the App Store. necessary if you want to change the business model from paid to in-app p.

Your Apps and the Future of macOS Security

More sandboxing and system apps data protection.

New stamp on signed Apps to reduce malware.

Introducing Create ML

Machine learning in Swift and Xcode

Training for Images, Text, Tabular data

Data gathering, training, and evaluation.

Image classification:

Data gathering: Dictionaries or Data source from folders and named files

Training: can use transfer learning on top of a model shipped with the OS. You just add your smaller set of data.

All from a playground. From a drag&drop UI or calling methods which helps for automation.

Text natural language. Text classification and word tagging.

Text classification: sentiment analysis, spam detection, topic analysis.

Data: label directories, csv.

Training: no need to detect language or tokenize, just give raw text with labels and they do the rest.

Tabular data.

MLDataTable, rows are observations and columns features and target.

CSV, JSON, Dictionaries in code.

Many operations in DataTable.

Different training algorithms and even an automatic algorithm picker.

Advances in Research and Care Frameworks

So great that Apple continues to move HealthKit, ResearchKit and CareKit forward.

Being developed in the open and giving write access to the repos!

ResearchKit

UI following recent iOS style.

The forms look awesome, I wonder if we shouldn’t be using this for other UIKit apps :P

PDF viewer for different steps.

Active tasks

​ Health records

​ Auditive tests (with data from the EarPods and environment noise detection )

​ Speech recognition

​ Speech and Noise

​ Vision, Amsler Grid

Movement Disorder API, passive monitoring from the watch.

Engineering Subscriptions

Use server-to-server validation

Let user use in-app purchases without creating accounts by using the original transaction id

What's New in watchOS

The watch experience: Notifications, Complications and Siri Shortcuts

Show dynamic notifications even from notification center.

Grouped by thread, your dynamic notification can handle new notification in the same view.

Interactive notifications.

Action buttons adjustable at runtime.

Critical alerts, with haptic feedback, need special entitlement and permission from the user.

Quiet notifications don’t show up on the screen, they go directly to notification enter.

Audio background mode.

AVFoundation APIs allow to share audio code between iOS and watchOS.

Data will be displayed in the now playing complication.

Customizable set of buttons.

Audio control.

Other apps that are not audio apps but would like to show now playing can use the new Now Playing View.

Improved workouts sessions API.

Siri Shortcuts integrate with the Siri watch face.

RelevantShorcut can customize them with custom UI information, and giving hints to when they should show up.

Siri on the watch can still trigger user created shortcuts only with voice.

Getting the Most out of Playgrounds in Xcode

New Step by step execution.

Allows you to execute just one line of code in the same context without rerunning, keeping the same data in memory.

Great for tutorials and data exploration.

PlaygroundDisplayConvertable

Playgrounds on Projects or Workspaces check the build products directory to find compiled frameworks.

(Still it would be nice if there was some integration with SPM to import third party frameworks easily)

Accessing Health Records with HealthKit

Health Records is really amazing but only used in the US. I wish all countries would allow for this.

Accessing them uses a new privacy screen, different from the rest of HK.

New authorization type, every time you access the data it can prompt the user.

And as usual with HK, you don’t know if the user granted or not permission.

The Argonaout Project looks awesome, I hope institutions are more open on giving you this data in the near future.

What's New in tvOS 12

Password autofill

The remote detects the closer phone and asks the password there. Magic!

Trigger login automatically with the focus engine. One tap login.

Focus engine protools so Metal apps can leverage the focus system

New common UI patterns for UIKit apps.

Poster, Caption button, card and monogram.

Even a cardView, would be nice to have it in iOS. Android has one :p

Digit entry viewcontroller.

Strategies for Securing Web Content

Secure transport

HTTPS and WSS

Cross-origin lockdown

Sent proper headers to allow only known domains.

Safe cookies

Stop using UIWebView, it runs in the same process and is dangerous.

Integrating Apps and Content with AR Quick Look

QuickLook does all the job for you to display an AR model

Files, Mail, Messages, Notes, News and Safari already support it.

Website and Apps can also use it.

What's New in Cocoa for macOS

NSStringEnum and NSextensibleString_enum

now NSTypedextensibleEnum and NSSwiftbridgedtypedef

naming changes,

formalized protocols

formal soft deprecation

NSKeyArchived with secure coding by default and returns errors instead of exceptions

NSColor modifiers

CALayers by default!

usernotifications framework comes like in ios

NSTextView.performValidatedReplacement is pretty cool, insertions that behave like the user did it, including reusing the proper styles that are already in the document.

quickactions in finder and Touch Bar

automator contextual workflow

Introduction to Siri Shortcuts

Surface your app features in the system.

Search, Siri watch face, lock screen, and Siri UI itself.

User ads a custom trigger phrase.

Creating shortcut.

Define, donate and handle.

NSUserActivity vs. custom Intents

NSUserActivity is good for simple shortcuts that open something in the app and are already offered to Spotlight and Handoff.

Intents run inline without launching the app, include custom voice responses and UI and granular predictions

Custom intent editor.

Category, title, description, confirmation required, you define patterns with arguments. Seems pretty powerful.

iOS includes a developer setting to show recent NSUserActivity donations so it’s easy to debug.

Use requiredInfoKeys to specify only the required keys so the system can suggest things more often.

Specific intents for playing media and to suggest upcoming media.

Metal for OpenGL Developers

Introducing Dark Mode

Introducing MapKit JS

MapKit available for websites. Same libraries that Apple uses on their websites.

Similar APIs to its iOS counterpart.

New Ways to Work with Workouts

Just collect the data that you need.

Treat HealthKit as the source of truth.

WorkoutBuilder helps collecting and saving the data.

Workout recovery will relaunch your app after a crash if a session was running.

HKCumulativeQuantitySeriesSample will put together data instead of the high frequency samples

Subclass of HKQuantitySample

For high level stats keep using HKStatistics/Collection/Query but for data analysis use the new HKQuantitySeriesSample.

CarPlay Audio and Navigation Apps

MPPlayableContent remastered

Improved performance, faster startup, smoother animations and better communication with your app.

CarPlay framework to support maps in the car.

Canvas to draw the map and templates for the rest of interactive UI.

What’s New in Core MLPart 1

Xcode previews information about the model and generates and interface.

Use integrated frameworks like Vision and Natural Language.

Download the model on demand and compile it on device to reduce the size of the App.

iOS 12 models can have different bit sizes so they’re more space efficient. Quantization.

Convert it with the ML tools.

Interface to your app doesn’t change. Is a tradeoff between accuracy and size.

Combine different functions in 1 model.

For example to support different image resolutions with a single model. (range or specific dimensions)

New batch API to perform multiple predictions so the it uses better the hardware.

Custom layers, if the ML inference engine doesn’t support layers that a model needs you can provide the code for them. It can even be a metal shader to performance is still good.

Custom models, API doesn’t change but is more flexible to support advancements on the field.

What's New in Testing

Xcode 9.3 improvements in code coverage (performance, file size and accuracy)

Specify which targets you want coverage for.

Command line tool for automation.

New mode in scheme to avoid adding automatically tests in schemes.

Easily have different schemes for different tests

Randomization mode!

Parallel tests

Information in Xcode about time tests take.

Tests are parallelized by classes This makes breaking tests classes in smaller ones a better idea.

Put performance tests in their own bundle and disable parallelization.

Building for Voice with Siri Shortcuts

Donated activities appear in Settings fo the user to create a shortcut.

Custom responses, template strings: Confirmation, Success, Information, Error.

Xcode generates code automatically an IntentResponse subclass with appropriate methods.

Intents UI for custom UI in Siri, lock screen and search.

Donate UserActivity or custom Intents

Customize: Title, Subtitle, Image and suggested invocation phrase.

INShortcut API (for intents and user activities) to suggest actions that the user hasn’t done yet.

Add suggestion creation into your App. INUIAddVoiceShortcutViewController.

API to remove shortcuts and to see what shortcuts the user has created so you can show them on your Apps UI.

New Localization Workflows in Xcode 10

Before, Xcode gathered localized resources and created and xliff file.

New, Xcode localization catalog. .xclog. Supports all assets with contextual information.

Deferred Localized API for strings in Siri Shortcuts so the string is localized when is presented and not when is donated.

Introducing ClassKit

Developer setting in device to change between teacher and student mode.

What’s New in Core MLPart 2

Core ML tools has converters for the majority of training frameworks.

With tensforflow support for quantization and custom layers.

ML tools helps quantizing models and compare them to the original to check precision.

Conversion now supports custom layers.

Advanced Dark Mode

You can overwrite the appearance of specific views.

Appearance source can be assigned to custom views so they inherit the right appearance.

Do appearance work in the views update methods (updateConstraints, layout, draw or updateLayer)

effective appearance bestMatch API to change the behavior based on appearances. (add extra views for example)

Semantic materials

Custom materials using VisualEffectView

Use maskImage in VisualEffectView to get material style in custom shape views.

Vibrancy in NSAppearance, use it in your views with grayscale colors. Vibrancy is inherited in the view hierarchy, apply it in specific leaf views.

Background styles are now also semantic.

Don’t use name colors (.black) nor hardcoded colors (RGB).

COOMING SOON

The Qualities of Great Design

Managing Documents In Your iOS Apps

Siri Shortcuts on the Siri Watch Face

What’s New in User Notifications

Inside SwiftShot: Creating an AR Game

Image and Graphics Best Practices

Measuring Performance Using Logging

Swift Generics

High Performance Auto Layout

Using Grouped Notifications

A Guide to Turi Create

TextKit Best Practices

Introducing Natural Language Framework

Metal for Ray Tracing Acceleration

Intentional Design

Designing Fluid Interfaces

Metal for Game Developers

Practical Approaches to Great App Performance

What's New in App Store Connect

Creating Photo and Video Effects Using Depth

Building Faster in Xcode

Data You Can Trust

Creating Audio Apps for watchOS

What's New in LLVM

Optimizing Your App for Today’s Internet

The Life of a Button

Integrating with Photos on macOS

What's New in Managing Apple Devices

Creating Great AR Experiences

Creating Custom Instruments

Introducing Network.framework: A modern alternative to Sockets

Object Tracking in Vision

Embracing Algorithms

A Tour of UICollectionView

Core Data Best Practices

Vision with Core ML

Automating App Store Connect

Metal Shader Debugging and Profiling

VoiceOver: App Testing Beyond The Visuals

Optimizing App Assets

Metal for Accelerating Machine Learning

Better Apps through Better Privacy

Getting to Know Swift Package Manager

Core Image: PerformancePrototypingand Python

Advanced Debugging with Xcode and LLDB

What’s New in Energy Debugging

Understanding ARKit Tracking and Detection

Using Collections Effectively

Metal for VR

Create Your Own Swift Playgrounds Subscription

Wallet and Apple Pay: Creating Great Customer Experiences

Metal Game Performance Optimization

MusicKit on the Web

Deliver an Exceptional Accessibility Experience

Designing Notifications

Understanding Crashes and Crash Logs

HomeKit Deep Dive

Design Studio Shorts

Getting Ready for Business Chat

iOS Memory Deep Dive

What’s New in Safari and WebKit

Behind the Scenes of the Xcode Build Process

Adding Delight to your iOS App

Testing Tips & Tricks

UIKit: Apps for Every Size and Shape

What's New in Search Ads

If you liked this article please consider supporting me