I was wondering why not use SVG (or any other vector file format) files and render all the graphic on the device, instead of including tons of PNG at various resolutions.
Fuzzy reasons abound, but you do occasionally see this. One major often-complete-blocker that I've seen though has been ungood rendering, so results aren't always identical on all devices. You can't really rely on the device's possibly-weird implementation (who knows what the OEM did to it)(except on iThings probably), so you're forced to bundle an svg renderer that works reliably on all devices and has consistent behavior, and good luck finding a high quality lib that does that[1]. And, to be friendly to the designers, it has to be reasonably full-featured.
I've made few apps and so far nobody cared about it. I thought that hand tweaking was about 16x16 icons, but now we are using 1024x1024 icons sometimes, there's no reason to tweak anything there, really. Actually it was the other way around: I wrote some scripts which rendered SVG into multiple PNGs and I think that almost everyone does the same.
SVG stadnardisation and support is growing fast, notably because it is now part of HTML5 and has strong support in browsers, which are the most active software development needed for all kinds of apps.
SVG includes many tests to assert the conformance and is one of the best graphic standard in terms of standardization with very precise requirements. It's in fact easy to get the expected results on wide ranges of devices, and implementations are now very performant. they are implemented as reusable opensource libraries or directly now in core services of most OS distributions, and even part of mobile platforms. The hardware is now optimized to support almost all SVG graphic primitives. The standard is not born from nowhere, it has inherited the best practices initiated earlier in PostScript then OpenGL, DirectX, X11 and similar APIs, and references other related technologies that are also standardized: Unicode, OpenType, IEC color spaces. There are still ways to extend it, but the SVG standard defines the requirements needed to support many devices, small or large.
Only very old implementations will suffer from some quirks due to absence of support of newer features, but the graphic will still render correctly provided they followed the mandatory conformance rules for each version.
Remeber that SVG extensions are widely upward compatible: newer renderers will support all graphics built for conforming renderers based on older versions.
Note that SVG is not designed to get exactly equivalent results but results that are suitable for use on each device. This is absolutely not the case for bitmap graphics which are extremely hard to adapt, and waste lot of bandwidth.
The next step for SVG will be to be a bit less verbose, suing another base syntax than XML: JSON seems to be the best candidate as it offers significant performance improvements to XML parsers: this will appear because now SVG is standardized primarily not for its syntax but for its DOM and API, just like HTML now and more recently as well with CSS (which is also used by SVG).
And more impoprtantly the SVG standard is open, allowing all kinds of experimentations and implementations. No new feature appears in the standard before there's already been experimentations.
The only few things that SVG do not support completely for now:
* fine tuning fonts to handle more semantics and typographical effects
* better support to render on non-additive color spaces (SVG still depends to much on sRVG which does not print very well, and new display technologies are now using more than 3 color planes to extend their gamut); colorspace transforms, and masks are still better supported in PostScript.
* support for 3D rendering is still in alpha stage, but will come sooner or later.
* support for splines may later add other curves than just quadratic or cubic Beziers, and elliptical arcs: hyperbolic, sinusoidal, and other parametric curves should appear, with their own rules for their subsampling decompositions to match the expected resolutions on target devices
* lighting models are still too simple. But the OpenGL standard already drives what will appear.
* support for texels should also come for 3D rendering, including mipmaps and synthetic/parametric mipmaps
* integration of videos with SVG animations is still a challenge; given the neeed to also synchronize in time with audio, users events or scenarios for gameplay.
The HTML Canvas will drive the needs, as well as other W3C specifications related to accessibility, internationalization, matching user preferences. You cannot do the same thing with "stupid" bitmap graphics. In fact the birmap graphics will be internally converted to vector graphics for better rendering (this is already what printer drivers are doing to improve their results, you cannot do that only by subsampling, bicubic interpolations: you need geometric transforms, notably for adapting texts or symbolic designs such as maps): the individual "pixels" in bitmaps are really stupid objects that do not properly handle how they are geometrically linked to their surrounding pixels: there's a need to contextually hide or show some details, or improve some of them, vector fonts for Opentype are perfect examples of that need.
It definitely is improving, and it's fantastic to see the progress. I really enjoy SVG assets, and I'd love to see them mostly take over (obviously they'll never be 100%, nor should they).
But that still means it's years away from widespread support, because large companies can't rely on it for large user-bases, and feature-sets are still pretty widely varying in my experience. Some support animations, some complex gradients and masking, and huge variance in text rendering (which is absolutely essential to finely crafted images) cripples some of the most-valuable use-cases (internationalized images without making dozens or hundreds of near-duplicates).
It's just not there yet. I hope it gets there, but there are significant hurdles that don't seem to be getting a lot of attention.
Sure. So hand-tweak the SVG files per density that you need custom-handled. Like you already do for images / when exporting images. Same difference....
... except now you need to do it on the device too, to make sure it renders identically there. Otherwise the same problem!
---
For very small images it doesn't matter / png may be smaller, but small images with that quality don't typically add up to hundreds of megabytes.
Raster images are still faster and often you can make your png images very small and do stretching.
OTOH, people do vector stuff for many things, it's just they typically don't use SVG but something more efficient. XML parsing is not lightweight compared to a binary format or some sort of vector -> code translator speed wise.
The XML parser is not much a problem: the time to parse it is insignificant compared to the waste of resources to store and transport bitmap graphics.
Bitmaps will also never be able to adapt their content gracefully to user preferences and constraints. Bitmaps have no intelligence; they are only suitable for photography, but even in this case they lack information for proper rendering, so you will nver same twice the same bitmap across devices and rendering supports
Just compare what we had in the past with bitmap fonts (now almost abandoned except for text consoles) and how we appreciate now to have scalable vector fonts.
Bitmaps are extremely complicate to adapt to any layout, whereas SVG graphics can be now self-adaptative, to render only what is needed.
You could ship SVGs and lazily generate PNGs for the specific device. Once it's generated, you keep it cached. Although it might be a little slower on the initial run. I guess it depends on your SVG's complexity.
Webkit does a fine job with SVGs. You could draw it on a <canvas> element to get a rasterized view of the SVG in the desired size, and save the resulting image.
Modern smartphones have quite competent GPUs. They can run 3D games at 1080p/60fps. They could raymarch circles around an svg icon.
And if it wasn't enough, the renderings could be cached for later use.
TBH not many people consider SVG format worth the effort unless the graphic is made from scratch. 1) an organization rarely rationalizes having a UI dev spend 6 something hours either transforming an existing PNG to an SVG, or spending 12 hours compleitly replicate an existing design in photoshop only for it to be exportable as an SVG. I could be far fe5ching things but sometimes business or upper management get short sighted on the small things. Probably there is no rationalizarion in spending extra effort in having these graphics all svg format, thus improving your application's load and bandwidth utilization, vs pumping more funds towards other easier quicker solutions.
AFAIK, the majority of designs are created in photoshop or illustrator. No conversion nessecary. In any case, I would consider it a liability if I didn't have my designs in an editable format. What if tommorow I realized I wanted a variant in a different color? Or I decided I wanted to use a different background? If all I had was a bitmap logo, I would consider it worthwhile to have someone change it into a scalable format anyways, even if I wanted to transfer them as PNGs.
Also, numerous free tool exist to convert bitmaps to SVG. Using one of these, plus some manual smoothing and correction, I'm sure the conversion would take no more than half an hour.
So you just CPU-rasterize out to a buffer, and blit it to the screen. After the initial render, it's equivalent to the PNG. The application essentially decompresses its assets on launch.
May be SVG is not the best vector format after all. May be we need something simpler, where every item directly maps to GPU graphics calls, also binary would help.
GPUs have graphic calls anymore. At best they have fragment and pixel shaders.
Converting a vector graphic into a good shader is not trivial, compared to using the built in functions of texturing.
It gets even worse once you realize how limited older mobile GPUs are or what are the incompatible subsets of features supported at decent performance.
If you want something simpler that works with GPUs it needs to be exclusively built out of triangles and/or code that can operate on a single pixel independently of its neighbors.
Resolution independent formats are inherently anti-triangle and by the time you've hit triangles you already have a target resolution in mind.
Or put another way, GPUs really don't like vector graphics in general. That's not what they're built for or good at.
GPUs like vector graphics just fine; rasterizing vector graphics is literally what GPUs are designed to do. It's infinitely scalable vector graphics that they tend to dislike.
Once you've subdivided your curves and such into triangles/vertices/etc., the GPU ends up being a lot happier about its existence.
GPUs are not designed for vector graphics, they are designed for triangles and triangles only. Triangles are a subset of vector graphics, but are almost never the subset that people mean when they say "vector graphics."
Specifically vector graphics typically includes curves, which GPUs just don't do at all.
Once you've tessellated a curve into triangles you've already baked in a desired resolution. You can't have resolution independent vector graphics in a GPU friendly way.
Because the experience is subpar when that is done. It is impossible to create a vector image that will result in clean, unblurred strokes at arbitrary small pixel sizes. Don't even suggest using TrueType-style hinting, I don't think any designer would want to touch that with a 10 foot pole.
I would have said exactly the opposite: blurred strokes belong to bitmaps, you are still thinking that pixels are square over a regular area that can opnly be rotated by multiples of 90 degrees.
SVG graphics are clean, result in NO blur at all. They don't even need any Truetype-style hinting. You've probably looked only at early non-conforming implementations using bad approximations. But precise rendering algorithms for SVG are specified and fully tested (this is not the case for bitmaps whose support is much more problementic: see what happens to photos and videos when editing them! Never twice the same result, and lot of losses and artefacts everywhere, including bad undesired effects, and notably the moiré effects which are much worse and do not reproduce what natural scaling would do in your eyes, that are not fitted to a perfect rectangular grid).
So no, it's perfectly possible to create a vector image that results in unblurred strokes at arbitrary pixel sizes. What you want is a geometric transform to exhibit more some details: it is in fact impossible to do that with bitmaps except by sending variants tuned for multiple resolutions: you do that for a limited number of resolution and assume it is enough, but this is not the case, so you add more, and finally you have enormous images that are widely redundant with each other, and better represented by converting their 2D model to 3D and an adaptive 3D-to2D projection. you'll do that easily with vector graphics, not with bitmaps (or their extensions called "mipmaps" which are only approximations not really scalable because they're still all based on discretely sampled planes).
Truetype hinting serves to distort the curves beneficially (by being aware of the pixel grid) in order to produce more readable output at low resolutions. SVG graphics do not automagically scale down and stay readable.
The classic easiest to explain in words case is taking a capital letter O and turning it into more of an "elongated stop sign" shape at around 6 pixels of width. There is a tradeoff between remaining faithful to the original shape and creating high-contrast, readable shapes at low resolution. There is nothing impossible about scaling down the vector outlines of a TrueType font in a mathematically perfect way. You still need the hints to make the result more readable.
(I worked on the hinting software for 13 of Apple's System 7 font faces and about 3/4 of Windows 3.1 launch fonts.)
> SVG graphics are clean, result in NO blur at all. They don't even need any Truetype-style hinting. You've probably looked only at early non-conforming implementations using bad approximations.
This huge wall of text but apparently you don't understand the subject matter all that deeply. Why wouldn't SVG need hinting? Truetype needs hinting and it's a vector format as well.
(Philippe always posts walls of text, even on the SVG mailing list; seems to be their thing.)
Hinting as done in TrueType is probably overkill for most purposes. But icons are often not displayed in arbitrary sizes, but rather in one of a few known-in-advance sizes. It's not hard to tweak the paths and shapes to fit to the pixel grid for all those. Also, as you approach higher pixel densities, it becomes much less important; basically you may just have to make sure that the smallest sizes fit on whole pixels and that's it.
Long ago I've automated asset generation in different sizes from SVG for an Android app I worked on and even without caring (much¹) about a pixel grid the results were good enough not to need tweaking.
_______________
¹ When hand-writing SVGs I tend to care about integer coordinates simply to not lose my mind.
I might agree that for very small icons (eg: targeting a ~VGA or 800x600 to maybe 1024x76 phone screen) hand painted bitmap graphics might be best... but shouldn't it be enough to ship 16x16 (or whatever) icons along with vector graphics for high-resolution screens? And if it's too slow to live-render the graphics, how about rendering custom icon sets (etc) on install - using the gpu, and then cache on the device?
SVGs are a problem because they don't have a "preferred size" parameter. On Xcode you can use PDF files for icons (which are much like SVG, but state "all my points are relative to the size WxH). Once you have that you can make pixel-perfect icons that are always rendered sharply.
The catch is that Xcode compiles all those PDFs to PNGs before bundling them into the .ipa, so in the end you have a bitmap either way...
pcwalton's project Pathfinder (https://github.com/pcwalton/pathfinder) may eventually lead to an efficient GPU-based vector renderer, which would make this feasible.
I mean, it's feasible now; plenty of folks use vector rendering: Caching the result makes the CPU rendering much easier to deal with. The GPU would really help more with animations and dynamically constructed vectors.
In any case, I suspect something like lyon (https://github.com/nical/lyon) would be a better fit than pathfinder, which is both a) not super portable and b) very focused on rendering text, which it does very well.
Thanks for indicating lyon, that looks very nice. I mentioned Pathfinder because path rendering on GPU is important on mobile devices (battery and heat concerns).
As of iOS 11/Xcode 9 vector images are supported which is quite nice. It may still convert to PNG for backwards compatibility though, I’m not sure if App thinning discards the PNG for iOS 11 devices.
Android starts to use it more and more. At least where it makes sense : icons and simple illustrations.
From discussions with iOS colleagues it seems that Android has seen a way stronger push for vector assets.
Probably because from day one the OS and SDK has been designed for unknown displays at compile time.
I'm not sure about iOS, but on Android the reason is the cost penalty of rasterizing larger icons and images on the fly. There's also some fragmentation with differing levels of SVG support on different OS levels. As a result, SVG is limited.
"some fragmentation..." That's like calling the ocean a bit wet! The experience of learning to code on Android revolves a great deal around dealing with that fragmentation.
I was specifically referring to the level of SVG support, which is constant across given OS versions, not per device like the fragmentation in old camera hardware abstraction layer implementations.
In that sense, the only reason there would be more fragmentation in SVG support on Android than iOS is that it can be harder to upgrade Android devices that did not come from Google.
As someone who has spent a lot of time with SVG, I can assure you there are very wide discrepancies in how the same file renders even using different tools on the same platform, let alone cross platform.
However, they usually are designed around a target size. And it's quite possible that, scaled too small, much of the detail is lost. Hence why you'd still want different vector files for different resolutions.
Considering the topic of this post (apps), I would argue assets are usually designed around a physical size of the image; even when you have multiple target display sizes, it becomes a part of the requirements to design assets the way that looks decent on all phones. Moreover, I find this whole branch of discussion a bit funny because assets are already designed to be high res and later downscaled for target dimensions, so simply using vectors literally just pushes that task to phone instead of build server.
