WWDC 18 session notes
As every year after WWDC is time to sit down and watch the videos of the sessions! I wrote last year about How I watch the WWDC videos and this year I’m doing a similar thing. I still use the unofficial WWDC for macOS to download the videos and watch them at more than 2x in VLC. This time I decided to use a single note in Notes app to keep my notes of the sessions and to mark the videos I’ve seen. And because everything is in a single note it makes it really easy to share, so I decided to share my notes with you!
This is a copy of the raw notes that I keep, so it may contain some mistakes or typos. It’s also not a full fledged summary of the talks but instead it reflects my interests on the topics and what I thought would be interesting to have as a quick reference. Some talks may not have notes because I was not interested on the topic (or because I watched them at midnight 😅).
2018-06-10: I’m sharing them without having watched all the sessions yet. 101 videos are not watched that quick :P
What’s New in Swift
Everything was known thanks to being Open Source! 🎉
Creating Apps for a Global Audience
Font cascade lost for when a font doesn’t support a language script.
Use proper api to scale fonts based on dynamic sizes
Localised emphasis should be done with bold or color changes in words instead of cursive as not all languages have cursives.
Introducing Podcast Analytics
iTunes U content in Podcast app.
Modern Podcast tags: Trailer, Season, Bonus, Episode Number
Podcasts for Apple Watch.
Chapter support for mp3 including chapter art.
Group podcast notifications.
Anonymous analytics.
What’s New in Cocoa Touch
DataSource prefetching is useful to avoid lost frames on cellForRow but iOS 11 could be asking for prefetch at same time was requesting a cell, iOS 12 is smarted and doesn’t do that so the CPU prioritizes displaying the cell
iOS 12 passes UIKit scrolling info to the CPU so it can rump up faster and on time to avoid dropping frames.
Memory allocation can have impact on performance, if big chinks are asked and there is no free memory system will give it from other apps. First, users will see other apps get closer when they don’t expect it, but your app will have a performance it while the system goes and reclaims that memory. so yeah, gg for random allocators
iOS 12 will configure image backing stores automatically to reduce the memory usage, you can configure it manually if you know what you want (grayscale vs deep color)
Autolayout internal changes, faster by default.
UIKit swiftyfied!
Name-spacing and nesting
free functions as methods
Rect, size, edges and other types now conform to Codable!
Notifications actions can now be dynamic and changed at runtime
Notifications can have live interactive views
Grouping respects thread identifier
Deep-link into your app notifications settings for granular controls
Apps can save password in login screens into keychain!
Password generation can be customized to follow your app requirements
Siri shortcut following NSUserActivity
Custom Siri intents
Or let users create a shortcut!
Using Accelerate and simd
Live Screen Broadcast with ReplayKit
Can check for is being recorded hide sensitive information
I Have This Idea For An App…
Tutorial for making an app ¯\(ツ)/¯
Automatic Strong Passwords and Security Code AutoFill
Autofill:
associated domain, done with universal links
Username content type
Password content type
WKWebview login autofill
Save credentials after login:
Automatic, based on content type, fields being removed from the hierarchy and cleared
For suggesting strong password user newPassword content type
Custom password rules in the textfield
Check with custom password validation tool website
tag one time code for sms codes
federated authentication, SSO
ASWebAuthenticationSession, shares cookies with Safari
Measuring and Optimizing HLS Performance
Streaming stuff ¯\(ツ)/¯
What’s New in ARKit 2
seems easy and cool
the demo of playing a video when detecting a picture is nice
Best Practices and What’s New with In-App Purchases
Subscriptions Introductory discounts
Free trials for non-subscriptions apps, kind of weirdly faked with multiple in-app purchases
StoreReviewController is limited but doesn’t avoid asking to often itself until it reaches the limits,
so you want to keep track of every requests, rate-limited yourself and don’t ask in the wrong moments.
app store URL link maker now can add an action to go to the review page directly
sandbox environment, compresses time of subscriptions 1 year - 1 hour, can request expired subscriptions or even simulate when a kid asks to buy to a parents phone.
sandbox account in the phone is different now! you don’t need to logout from your real account!
receipt has a lot of info about in-app purchases and original purchases from the App Store. necessary if you want to change the business model from paid to in-app p.
Your Apps and the Future of macOS Security
More sandboxing and system apps data protection.
New stamp on signed Apps to reduce malware.
Introducing Create ML
Machine learning in Swift and Xcode
Training for Images, Text, Tabular data
Data gathering, training, and evaluation.
Image classification:
Data gathering: Dictionaries or Data source from folders and named files
Training: can use transfer learning on top of a model shipped with the OS. You just add your smaller set of data.
All from a playground. From a drag&drop UI or calling methods which helps for automation.
Text natural language. Text classification and word tagging.
Text classification: sentiment analysis, spam detection, topic analysis.
Data: label directories, csv.
Training: no need to detect language or tokenize, just give raw text with labels and they do the rest.
Tabular data.
MLDataTable, rows are observations and columns features and target.
CSV, JSON, Dictionaries in code.
Many operations in DataTable.
Different training algorithms and even an automatic algorithm picker.
Advances in Research and Care Frameworks
So great that Apple continues to move HealthKit, ResearchKit and CareKit forward.
Being developed in the open and giving write access to the repos!
ResearchKit
UI following recent iOS style.
The forms look awesome, I wonder if we shouldn’t be using this for other UIKit apps :P
PDF viewer for different steps.
Active tasks
Health records
Auditive tests (with data from the EarPods and environment noise detection )
Speech recognition
Speech and Noise
Vision, Amsler Grid
Movement Disorder API, passive monitoring from the watch.
Engineering Subscriptions
Use server-to-server validation
Let user use in-app purchases without creating accounts by using the original transaction id
What’s New in watchOS
The watch experience: Notifications, Complications and Siri Shortcuts
Show dynamic notifications even from notification center.
Grouped by thread, your dynamic notification can handle new notification in the same view.
Interactive notifications.
Action buttons adjustable at runtime.
Critical alerts, with haptic feedback, need special entitlement and permission from the user.
Quiet notifications don’t show up on the screen, they go directly to notification enter.
Audio background mode.
AVFoundation APIs allow to share audio code between iOS and watchOS.
Data will be displayed in the now playing complication.
Customizable set of buttons.
Audio control.
Other apps that are not audio apps but would like to show now playing can use the new Now Playing View.
Improved workouts sessions API.
Siri Shortcuts integrate with the Siri watch face.
RelevantShorcut can customize them with custom UI information, and giving hints to when they should show up.
Siri on the watch can still trigger user created shortcuts only with voice.
Getting the Most out of Playgrounds in Xcode
New Step by step execution.
Allows you to execute just one line of code in the same context without rerunning, keeping the same data in memory.
Great for tutorials and data exploration.
PlaygroundDisplayConvertable
Playgrounds on Projects or Workspaces check the build products directory to find compiled frameworks.
(Still it would be nice if there was some integration with SPM to import third party frameworks easily)
Accessing Health Records with HealthKit
Health Records is really amazing but only used in the US. I wish all countries would allow for this.
Accessing them uses a new privacy screen, different from the rest of HK.
New authorization type, every time you access the data it can prompt the user.
And as usual with HK, you don’t know if the user granted or not permission.
The Argonaout Project looks awesome, I hope institutions are more open on giving you this data in the near future.
What’s New in tvOS 12
Password autofill
The remote detects the closer phone and asks the password there. Magic!
Trigger login automatically with the focus engine. One tap login.
Focus engine protools so Metal apps can leverage the focus system
New common UI patterns for UIKit apps.
Poster, Caption button, card and monogram.
Even a cardView, would be nice to have it in iOS. Android has one :p
Digit entry viewcontroller.
Strategies for Securing Web Content
Secure transport
HTTPS and WSS
Cross-origin lockdown
Sent proper headers to allow only known domains.
Safe cookies
Stop using UIWebView, it runs in the same process and is dangerous.
Integrating Apps and Content with AR Quick Look
QuickLook does all the job for you to display an AR model
Files, Mail, Messages, Notes, News and Safari already support it.
Website and Apps can also use it.
What’s New in Cocoa for macOS
NS_String_Enum and NS_extensible_String_enum
now NS_Typed_extensible_Enum and NS_Swift_bridged_typedef
naming changes,
formalized protocols
formal soft deprecation
NSKeyArchived with secure coding by default and returns errors instead of exceptions
NSColor modifiers
CALayers by default!
usernotifications framework comes like in ios
NSTextView.performValidatedReplacement is pretty cool, insertions that behave like the user did it, including reusing the proper styles that are already in the document.
quickactions in finder and Touch Bar
automator contextual workflow
Introduction to Siri Shortcuts
Surface your app features in the system.
Search, Siri watch face, lock screen, and Siri UI itself.
User ads a custom trigger phrase.
Creating shortcut.
Define, donate and handle.
NSUserActivity vs. custom Intents
NSUserActivity is good for simple shortcuts that open something in the app and are already offered to Spotlight and Handoff.
Intents run inline without launching the app, include custom voice responses and UI and granular predictions
Custom intent editor.
Category, title, description, confirmation required, you define patterns with arguments. Seems pretty powerful.
iOS includes a developer setting to show recent NSUserActivity donations so it’s easy to debug.
Use requiredInfoKeys to specify only the required keys so the system can suggest things more often.
Specific intents for playing media and to suggest upcoming media.
Metal for OpenGL Developers
Introducing Dark Mode
Introducing MapKit JS
MapKit available for websites. Same libraries that Apple uses on their websites.
Similar APIs to its iOS counterpart.
New Ways to Work with Workouts
Just collect the data that you need.
Treat HealthKit as the source of truth.
WorkoutBuilder helps collecting and saving the data.
Workout recovery will relaunch your app after a crash if a session was running.
HKCumulativeQuantitySeriesSample will put together data instead of the high frequency samples
Subclass of HKQuantitySample
For high level stats keep using HKStatistics/Collection/Query but for data analysis use the new HKQuantitySeriesSample.
CarPlay Audio and Navigation Apps
MPPlayableContent remastered
Improved performance, faster startup, smoother animations and better communication with your app.
CarPlay framework to support maps in the car.
Canvas to draw the map and templates for the rest of interactive UI.
What’s New in Core MLPart 1
Xcode previews information about the model and generates and interface.
Use integrated frameworks like Vision and Natural Language.
Download the model on demand and compile it on device to reduce the size of the App.
iOS 12 models can have different bit sizes so they’re more space efficient. Quantization.
Convert it with the ML tools.
Interface to your app doesn’t change. Is a tradeoff between accuracy and size.
Combine different functions in 1 model.
For example to support different image resolutions with a single model. (range or specific dimensions)
New batch API to perform multiple predictions so the it uses better the hardware.
Custom layers, if the ML inference engine doesn’t support layers that a model needs you can provide the code for them. It can even be a metal shader to performance is still good.
Custom models, API doesn’t change but is more flexible to support advancements on the field.
What’s New in Testing
Xcode 9.3 improvements in code coverage (performance, file size and accuracy)
Specify which targets you want coverage for.
Command line tool for automation.
New mode in scheme to avoid adding automatically tests in schemes.
Easily have different schemes for different tests
Randomization mode!
Parallel tests
Information in Xcode about time tests take.
Tests are parallelized by classes This makes breaking tests classes in smaller ones a better idea.
Put performance tests in their own bundle and disable parallelization.
Building for Voice with Siri Shortcuts
Donated activities appear in Settings fo the user to create a shortcut.
Custom responses, template strings: Confirmation, Success, Information, Error.
Xcode generates code automatically an IntentResponse subclass with appropriate methods.
Intents UI for custom UI in Siri, lock screen and search.
Donate UserActivity or custom Intents
Customize: Title, Subtitle, Image and suggested invocation phrase.
INShortcut API (for intents and user activities) to suggest actions that the user hasn’t done yet.
Add suggestion creation into your App. INUIAddVoiceShortcutViewController.
API to remove shortcuts and to see what shortcuts the user has created so you can show them on your Apps UI.
New Localization Workflows in Xcode 10
Before, Xcode gathered localized resources and created and xliff file.
New, Xcode localization catalog. .xclog. Supports all assets with contextual information.
Deferred Localized API for strings in Siri Shortcuts so the string is localized when is presented and not when is donated.
Introducing ClassKit
Developer setting in device to change between teacher and student mode.
What’s New in Core MLPart 2
Core ML tools has converters for the majority of training frameworks.
With tensforflow support for quantization and custom layers.
ML tools helps quantizing models and compare them to the original to check precision.
Conversion now supports custom layers.
Advanced Dark Mode
You can overwrite the appearance of specific views.
Appearance source can be assigned to custom views so they inherit the right appearance.
Do appearance work in the views update methods (updateConstraints, layout, draw or updateLayer)
effective appearance bestMatch API to change the behavior based on appearances. (add extra views for example)
Semantic materials
Custom materials using VisualEffectView
Use maskImage in VisualEffectView to get material style in custom shape views.
Vibrancy in NSAppearance, use it in your views with grayscale colors. Vibrancy is inherited in the view hierarchy, apply it in specific leaf views.
Background styles are now also semantic.
Don’t use name colors (.black) nor hardcoded colors (RGB).
The Qualities of Great Design
Managing Documents In Your iOS Apps
Siri Shortcuts on the Siri Watch Face
What’s New in User Notifications
Inside SwiftShot: Creating an AR Game
Image and Graphics Best Practices
Measuring Performance Using Logging
Swift Generics
High Performance Auto Layout
Using Grouped Notifications
A Guide to Turi Create
TextKit Best Practices
Introducing Natural Language Framework
Metal for Ray Tracing Acceleration
Intentional Design
Designing Fluid Interfaces
Metal for Game Developers
Practical Approaches to Great App Performance
What’s New in App Store Connect
Creating Photo and Video Effects Using Depth
Building Faster in Xcode
Data You Can Trust
Creating Audio Apps for watchOS
What’s New in LLVM
Optimizing Your App for Today’s Internet
The Life of a Button
Integrating with Photos on macOS
What’s New in Managing Apple Devices
Creating Great AR Experiences
Creating Custom Instruments
Introducing Network.framework: A modern alternative to Sockets
Object Tracking in Vision
Embracing Algorithms
A Tour of UICollectionView
Core Data Best Practices
Vision with Core ML
Automating App Store Connect
Metal Shader Debugging and Profiling
VoiceOver: App Testing Beyond The Visuals
Optimizing App Assets
Metal for Accelerating Machine Learning
Better Apps through Better Privacy
Getting to Know Swift Package Manager
Core Image: PerformancePrototypingand Python
Advanced Debugging with Xcode and LLDB
What’s New in Energy Debugging
Understanding ARKit Tracking and Detection
Using Collections Effectively
Metal for VR
Create Your Own Swift Playgrounds Subscription
Wallet and Apple Pay: Creating Great Customer Experiences
Metal Game Performance Optimization
MusicKit on the Web
Deliver an Exceptional Accessibility Experience
Designing Notifications
Understanding Crashes and Crash Logs
HomeKit Deep Dive
Design Studio Shorts
Getting Ready for Business Chat
iOS Memory Deep Dive
What’s New in Safari and WebKit
Behind the Scenes of the Xcode Build Process
Adding Delight to your iOS App
Testing Tips & Tricks
UIKit: Apps for Every Size and Shape
What’s New in Search Ads