Live Blogging From SymfonyLive London 2019

September 13th, 2019

Inspired by Matt Stauffer's live blogging of the keynote at Laracon US, I’m going to do the same for the sessions that I’m attending at SymfonyLive London 2019...

Keynote (Back to the basics)

Embrace the Linux philosophy

• How we grow the Symfony ecosystem. Built abstracts.
• HttpFoundation, HttpKernel
• Moved to infrastructure
• A few abstractions on top of PHP. Improved versions of PHP functions (dump and var_dump)
• Started a add higher level abstractions (e.g. Mailer), built on the lower ones.
• Recently worked on PHPUnit assertions. Mailer in Symony 4.4. Can test if an email is sent or queued

Building flexible high-level abstractions on top of low-level ones

What's next?

• Mailer announced in London last year. New component.
• System emails? e.g. new customer, new invoice.
• Symfony Mailer = Built-in responsive, flexible, and generic system emails
  • Twig with TwigExtraBundle
  • Twig inky-extra package (Twig 1.12+)
  • Zurb Foundation for Emails CSS stylesheet
  • Twig cssinliner-extra package (Twig 1.12+)
  • Optimised Twig layouts
• SystemEmail class extends templated email
• Can set importance,
• Customisable
• Always trying to keep flexible, so things can be overidden and customised

Sending SMS messages

• new Texter and SmsMessage class for sending SMS messages
• Same abstraction as emails, but for SMS messages
• Based on HttpClient + Symfony Messenger and third-party providers (Twilio and Nexmo) twilio:// and nemxo://
• Can set via transport $sms->setTransport('nexmo')
• Extend the SystemEmail and do what you want
• Failover

Sending Messages

• Create ChatMessage
• Telegram and Slack
• $message->setTransport('telegram'), $bus->dispatch($message)
• Send to Slack and Telegram
• SlackOptions and TelegramOptions for adding emojis etc
• Common transport layer TransportInterface, MessageInterface
• Failover - e.g. if Twilio is down, send to Telegram

New component - SymfonyNotifier

• Channels - email, SMS, chat
• Transport, slack, telegram, twilio
• Create a notification, arguments are message and transports (array)
• Receiver
• Customise notifications, InvoiceNotification extends Notification. getChannels
  • Override default rendering
  • ChatNotificationInterface - asChatMessage()
• Semantic configuration
  • composer req twilio-notifier telegram-notifier
• Channels
  • Mailer
  • Chatter
  • Texter
  • Browser
  • Pusher (iOS, Android, Desktop native notifications)
  • Database (web notification centre)
  • A unified way to notify Users via a unified Transport layer
• Each integration is only 40 lines of code

What about a SystemNotification?

• Autoconfigured channels
• new SystemNotification, Notifier::getSystemReceivers
• Importance, automatically configures channels
• Different channels based on importance
• ExceptionNotification - get email with stack trace attached

Notifier

• send messages via a unified api
• send to one or many receivers
• Default configu or custom one

 How can we leverage this new infrastructure?

• Monolog NotifierHandler - triggered on Error level logs
• Uses notified channel configuration
• Converts Error level logs to importance levels
• Configurablelike other Notifications
• 40 lines of code

• Failed Messages Listener - 10 lines of glue code

• Experimental component in 5.0

• Can't in in 4.4 as it's a LTS version
• First time an experimental component is added
• Stable in 5.1

Queues, busses and the Messenger component (Tobias Nyholm)

• Stack is top and buttom - Last-in, first-out
• Queue is back and front - last in, first out

2013

• Using Symfony, used 40 or 50 bundles in a project - too much information!
• Used to copy and paste, duplicate a lot of code
• Testing your controllers - controllers as services?
• Controllers are 'comfortable'
• Tried adding CurrentUserProvider service to core, should be passed as an argument. Cannot test.
• 'Having Symfony all over the place wasn't the best thing' - when to framework (Matthias Noback)
  • Hexagonal architecture
  • Keep your kernel away from infrastructure. Let the framework handle the infrastructure.
• Controller -> Command -> Command Bus -> CommandHandler

What did we win?

• Can leverage Middleware with a command bus
• Queues as a service (RabbitMQ)
• Work queue - one producer, multiple consumers
• Queues should be durable - messages are also stored on disk, consumers should acknowledge a message once a message is handled
• Publish/subscribe
  • Producer -> Fanout/direct with routing (multiple queues) -> multiple consumers
• Topics - wildcards

2016

• New intern. Understand everything, 'just PHP'. Plain PHP application, not 'scary Symfony'

Symfony Messenger

• composer req symfony/messager - best MessageBus implementation
• Message -> Message bus -> Message handler
• Message is a plain PHP class
• Handler is a normal PHP class which is invokable
• messenger:message_hander tag in config
• Autowire with MessageHandlerInterface
• What if it takes 20 seconds to send a message? Use asynchronous.
• Transports as middleware (needs sender, receiver, configurable with DSN, encode/decode). MESSENGER_DSN added to .env
• Start consumer with bin/console messager:consume-messages. Time limit with --time-limit 300
• PHP Enqueue - production ready, battle-tested messaging solution for PHP

Issues

• Transformers, takes an object and transforms into an array - FooTransformer implements TransformerInterface.
• Don't break other apps by changing the payload.

Multiple buses

• Command bus, query bus, event bus
• Separate actions from reactions

Envelope

• Stamps for metadata - has the item been on the queue already?

Failures

• Requeue, different queue or same queue after a period of time
• Failed queue 1 every minute, failed queue 2 every hour - temporary glitches or a bug?

Creating entities

• What if two users registered at the same tiem? Use uuids rather than IDs.

• Symfony validation - can be used on messages, not just forms.

• Cache everything

  • Option 1: HTTP request -> Thin app (gets responses from Redis) -> POST to queue. Every GET request would warm cache
  • Option 2: HTTP request -> Thin app -> return 200 response -> pass to workers

• Tip: put Command and CommandHandlers in the same directory

 HttpClient (Nicolas Grekas)

• new symfony component, released in may
• Httpclient contracts, separate package that contains interfaces
  • Symfony
  • PHP-FIG
  • Httplug
• HttpClient::create(). $client->get()
• JSON decoded with error handling
• Used on symfony.com website (#1391). Replaces Guzzle Client for HttpClientInterface
• Object is stateless, Guzzle is not. Doesn't handle cookies, cookies are state
• Remove boilerplate - use toArray()
• Options as third argument - array of headers, similar to Guzzle

What can we do with the Response?

• getStatusCode(): int
• getHeaders(): array
• getContent(): string
• toArray(): array
• cancel(): void
• getInfo(): array - metadata
• Everything is lazy!
• 80% of use-cases covered

What about PSR-18?

• Decorator/adapter to change to PSR compatible
• Same for Httplug

What about the remaining 20%?

• Options are part of the abstraction, not the implementation

Some of the options

• timeout - control inactivity periods
• proxy - get through a http proxy
• on_progress - display a progress bar / build a scoped client
• base_url - resolve relative URLS / build a scoped client
• resolve - protect webhooks against calls to internal endpoints

• max_redirects - disable or limit redirects

• Robust and failsafe by default

• Streamable uploads - $mimeParts->toIterable().

• donwload a file

  foreach ($client->stream($response) as $chunk) {
    // ...
  }
  

• Responses are lazy, requests are concurrent

• Asychronus requests. Reading in network order

foreach ($client->stream($responses) as $response => $chunk) {
    if ($chunk->isLast()) {
        // a $response completed
    } else {
        // a $response's got network activity or timeout
    }
}

• 379 request completed in 0.4s!
• Stream has second argument, max number of seconds to wait before yielding a timeout chunk
• ResponseInterface::getInfo() - get response headers, redirect count and URL, start time, HTTP method and code, user data and URL
  • getInfo('debug') - displays debug information

The components

• NativeHttpClient and CurlHttpClient
  • both provide
  • 100% contracts
  • secure directs
  • extended (time) info
  • transparent HTTP compression and (de)chunking
  • automatic HTTP proxy configuration via env vars

NativeHttpClient

• is most portable, works for everyone
• based on HTTP stream wrapper with fixed redirect logic
• blocking until response headers arrive

CurlHttpClient

• Requires ext-curl with fixed redirection logic
• Multiplexing response headers and bodies
• Leverages HTTP/2 and PUSH when available
• Keeps connections open also between synchronous requests, no DNS resolution so things are faster

Decorators

• ScopingHttpClient - auto-configure options based on request URL
• MockHttpClient - for testing, doesn't make actual HTTP requests
• CachingHttpClient - adds caching on a HTTP request
• Psr18Client
• HttplugClient
• TraceableHttpClient

Combining

FrameworkBundle/Autowiring

framework:
  http_client:
    max_host_connections: 4
    deault_options:
      # ....
  scoped_client:
    # ...

HttpBrowser

• HttpClient + DomCrawler + CssSelector + HttpKernel + BrowserKit
• RIP Goutte!

Coming in 4.4...

• max_duration
• buffer based on a callable
• $chunk->isInformational()
• $response->toStream()
• Async-compatible extensibility, when decoration is not enough

composer req symfony/http-client

Symfony Checker is coming (Valentine Boineau)

• Static analysis tool for Symfony
  • Does a method exist?
  • Is it deprecated?
• insight.symfony.com
• @symfonyinsight
• Released soon

 Differences

• Specialise in Symfony - can see more relevant things
• Different interface to other services

Feeling unfulfilled by SPA promises? Go back to Twig (Dan Blows)

A way on the front-end JS, CSS, images at the beginning of the request, sends a HTTP request (XHR/AJAX) to the back-end

Why SPAs?

• A way on the front-end JS, CSS, images at the beginning of the request, sends a HTTP request (XHR/AJAX) to the back-end
• no full page refresh
• Supposed to be much quicker
• 'Right tool for the job' - JS on the front-end, PHP on the back-end
• Division of responsibility == faster development
• Reusable API - Api -> Mobile App and SPA - easy to add another consumer
• Easier to debug?

Why not SPAs?

• Lots of HTTP requests (400 to load the initial page on one project) == slow front end
• Blurred responsibilities == tightly coupled teams
• harder to debug, bugs fall between systems and teams. Huge gap between front-end and back-end, passing responsibilites.
• You can fix these problems in SPAs, but is it worth it?
  • Examples of good SPAs - Trello, Flickr

Using Twig as an alternative to an SPA?

Faster UI - Try and figure out where the problem is.

If you're trying to speed things up, find out where the problem is.

• Browser tools
• Web Debug Toolbar
• Blackfire
• Optimise and monitor

Speed up Twig

• Speeding up Symfony
• ext/twig (PHP5 only, not PHP 7)
• Store compiled templates in Opcache, make sure it's enabled
• Render assets though the webserver (assetic not running all the time)

Edge side includes

• Component cached differently to the rest of the page
• Varnish/Nginx
• render_esi
• News block that caches frequently, rest of the page

HTTP/2 with Weblink

• slow finding CSS files to load - 'push' over CSS files, doesn't need to wait
• preload() - https://symfony.com/doc/current/web_link.html

Live updating pages

• Instantly update when sports results are updated, news articles are added
• Mercure - https://github.com/symfony/mercure
• LiveTwig - whole block or whole section, and live update render_live
• Turbolinks - replace whole body, keeps CSS and JS in memory. Merges new stuff in. helthe/turbolinks
• ReactPHP - shares kernel between requests

Writing better code with Twig

• Keep templates simple. Avoid spaghetti code, only about UI. HTML or small amounts of Twig.

• Avoid delimeter chains

  • Bad:blog_post.authors.first.user_account.email_address
  • Good
  • Less brittle, slow

• Filters

  • Use filters to be precise
  • Custom filters
  • Avoid chains. Can cause odd results. Create a new filter in PHP
• Functions
  • Write your own functions
  • Simpler templates
  • Get data, can use boolean statements

• Components

  • Break a page into components rather than one large page
  • include()
  • Use only to only pass that data. less tightenly coupled.
  • render calls the whole of Symfony, boots Kernel, can be expensive and slow
  • Loosely couple templates and controllers
  • Keep responses simple
  • What makes sense
  • if you need extra data in the template, get it in the template
  • View models
  • Mixed results
  • BlogPostViewModel
  • Can result in boilerplate code
  • Can be useful if the view model is different to the Entity
  • DRY
  • "Don't repeat yourself"

• Faster development

  • Separate UI tests from back-end tests. Different layers for different teams. People don't need to run everything if they are only changing certain things.

• Help your front end

  • Webpack - Encore
  • Type hinting in functions and filters, easier to debug
  • Logging
  • Friendly exceptions - help front-end devs by returning meaningful, readbale errors
  • Web Debug Toolbar and Profiler, provide training for toolbar and profilers
  • Twig-friendly development environment - Twig support in IDEs and text editors

SPAs are sometimes teh right solution. Why do they want to use it, can the same benefits be added with Twig?

3 most important points:

• Profile, identidy, optimise, monitor
• Loosely couple templates to your app code
• Help your front ends - put your front end developers first
• You don't need to use a SPA for single pages, use JavaScript for that one page. It doesn't need to be all or nothing.

BDD Your Symfony Application (Kamil Kokot)

• Applying BDD to Sylius
• 2 years since release of Sylius (Symfony 2 alpha)
• The business part is more important than the code part

What is BDD?

• Behaviour driven development. Combines TDD and DDD, into an agile methodology
• Encourages communication and creates shared understanding
• Living, executable documentation that non-programmers understand. Always correct.
• Feature file
  • Feature
  • Scenario - example of the behaviour for this feature. Simple, atomic. (e.g. I need a product in order to add it to a cart)
  • In order to...
  • Who gets the benefit?

 BDD in practice

• Feature: booking flight tickets
• Scenario: booking flight ticket for one person
  • Given there are the following flights...
  • When I visit '/flight/LTN-WAW'
  • Then I should be on '/flight/LTN-WAW'
  • Add I should see "Your flight has been booked." in "#result"
• In the BDD way - what is the business logic? What is the value for this scenario? What is the reason 'why', and who benefits from this?
  • We just need to know that there are 5 seats left on a flight
  • Talk and communicate about how the feature is going to work - not just developers
  • BDD aids communication
• Questions we can ask
  • Can we get a different outcome when the context changes?
  • When there was only one seat available
  • When there were no available seats
  • Can we get the same outcome when the event changes? Can we change 'When' and 'Then stays the same'
  • When it is booked for an adult and a child
  • When it is booked for an adult
  • Does anything else happen that is not mentioned?
  • Generate an invoice if a seat is booked
  • a pilot would like to get a notification that a seat was booked.
  • Figuring out the rules
  • Adults are 15+ years old
  • Children are 2-14 years old
  • Infants and children can only travel with an adult
  • We don't allow for overbooking
  • Translating rules into examples
  • Add a new scenario for each rule - e.g. don't allow over booking
    • "And the flight should be no longer available..."

Behat

• Used to automate and execute BDD tests, also SpecDDD
• maps steps to PHP code
• Given a context, when an event, then an outcome
• Domain Context, API context
• class implements Context, annotations for @Given, @When, @Then. allows for arguments and regular expressions
• Suites: change what code is executed, and what scenarios are executed. context and tags
• FriendsOfBehat SymfonyExtension - integrates Behat with Symfony
  • Contexts registered as Symfony services - inject dependencies, service as a context in Behat. Need to be 'public' for it to work
  • Reduces boilerplate code. Supports autowiring.
  • Zero configuration

Domain context

• Given verb matches @Given annotation. Same for When and Then.
• Transformers, type hint name string, return Client instance

API context

• inject FlightBookingService and KernelBrowser
• Use $this->kernelBrowser->request()
• Use assert() function wuthin @Then

Back to reality - how it's done with Sylius

• Business part applies to all context. Start talking about what needs to be done, start communicating
• Implement contexts for UI and API
• 12716 steps, 1175 scenarios, 8 min 8 sec, 2.4 scenarios /sec
• 12x faster than JS (17 min 48 sec, 0.19 scenario / sec)

• Treat test CI environment like production

  • Turn off debug settings, add caching
  • Enable OPcache

• Write features in a natural way

• Too many setup steps - merge steps. less visual debt. e.g. Create currency, zone and locale when creating a store
• Avoid scenarios that are too detailed. You should specify only what's important to this scenario.

Migrating to Symfony one route at a time (Steve Winter)

• New client with an old application, built in an old version of another framework with unusual dependency management, no tests, no version control and deploying via FTP. Done over a ~3 month period.

• Subscription based index of suppliers

• New requirements to implement by the client
• Our requirements: Needed a deployment process, make it testable, fix the build chain
• Solution attempt 1: Migrate to a new version of the current framework
  • Minor template and design changes were fine
  • Modifiy features, add new dependencies.
• Solution attempt 2: Upgrade to the latest version - same outcome due to multiple BC breaks (no semver), lots of manual steps
• Solution attempt 3: Symfony!
  • Semver! Backwards compatibility promise
  • Symfony app to run in parallel, Apache proxy rules and minor changes to the legacy app, added data transfer mechanisms
  • Anything new done in Symfony
  • Installed on the same server with it's own vhost but not publicly accessible
  • Deployed independently of legacy app

Apache proxy rules

Proxy /public to symfony app

Legacy app

• Shared cookie for single login between apps - user account details (name etc), session details (login time)

Added functionality

• Built in Symfony
• new proxy rules for new routes
• Add menu links to legacy app menu
• How do we show how many reminders are active?
  • Symfony based API called from the front-end

Migrating routes

• Rebuilt or extend in Symfony app
• Test and deploy, then update the apache config to add new proxy rules

A gotcha

• Legacy app uses CSRF
• Needed to track the token, added to shared cookie and pass through to the Symfony side

Storing data

• Both apps using the same data with different credentials
• Some shared tables, some tables are specific to each app

Remaining challenges

• User session management, still handled by legacy app
• Templating/CSS - two versions of everything
  • Next step: move all CSS to Symfony

Summary

• Add Symfony app, Apache proxy rules for routes
• User transfer mechanisms
• New functionality added in Symfony

Is this right for you?

It depends. Fine for a 'modest' size. Use a real proxy for larger scale apps, use different servers with database replication.

Closing Keynote: The fabulous World of Emojis and other Unicode symbols (Nicolas Grekas)

• ASCII. Still used today. Map between the first 128 numbers to characters. OK for UK and US.
• 256 numbers in Windows-1252 (character sets). Each country had their own set.
• It's legacy. 0.2% for Windows-1252. 88.8% for UTF-8 (Feb 2017)
• Unicode: 130k characters, 135 scripts (alphabets)
• Validation errors using native alphabet - e.g. invalid last name when submitting a form
• 17 plans, each square is 255 code points
• Emojis are characters, not images
• Gliph is a visual representation of a character
• From code points to bytes
  • UTF-8: 1,2,3 or 4 bytes
  • UTF16: 2 or 4 bytes
  • UTF-32: 4 bytes
• UTF-8 is compatible with ASCII
• Case sensitivity - 1k characters are concerned. One uppercase letter, two lower case variants. Turkish exception (similar looking letters that are different letters with different meanings). Full case folding.
• Collations - ordering is depends on the language. 'ch' in Spanish is a single character.
• Single number in unicode to represent accents. Combining characters.
• Composed (NFC) and decomposed (NFD) forms - normalisation for comparison
• Grapheme clusters - multiple characters, but one letter as you write it (separate characters for letters and accent)
• Emjois - combining characters. e.g. Combine face with colour. Different codes and character names. Also applies to ligatures. A way to combine several images together into one single visual representation.

unicode fundamentals

• uppercase, lowercase, folding
• compositions, ligatures
• comparistions - normalisations and collations
• segmentation: characters, words, sentences and hyphens
• locales: cultural conventions, translitterations
• identifiers & security, confusables
• display: direction, width

unicode in practice

• MySQL - utf*_*. SET NAMES utf8mb4 for security and storing emojis. Cannot store emojis with utf8

in php

• mb_*()
• iconv_*()
• preg_*()
• grapheme_*() normalizer_*()
• symfony/polyfill-* - pure PHP implementation
• Made a component - symfony/string - https://github.com/symfony/symfony/pull/33553
• Object orientated api for strings. Immutable value objects

• AbstractString

  • GraphemeString
  • Utf8String
  • BinaryString

• AbstractString - Methods to serialize, get length, to binary or grapheme or utf8

  • Methods for starts with, ends with, is empty, join, prepend, split, trim, title etc