There is a TMS tile data source, and we need to "simulate" a WMTS service. What should we do?
In this case, there are existing infrastructures or tools to solve the problem, such as various map servers, and the .net ecosystem also has open-source tools like tile-map-service-net5. The reason this issue is a problem lies in two constraints.
- The client used does not support loading XYZ/TMS format data and can only load WMS and WMTS format data.
- The data being used is in a pre-sliced TMS structure.
- The client is inconvenient to rely on external map servers.
Mimicking Resource Links#
Some familiar internet maps use XYZ or TMS methods, such as OSM, Google Map, and Mapbox. From the previous raster tiles to the now more common vector tiles, to use TMS to "mimic" the WMTS request format, we need to first understand what differences they have.
XYZ (slippy map tilename)#
- 256*256 pixel images
- Each zoom level is a folder, each column is a subfolder, and each tile is an image file named with the row
- The format is similar to
/zoom/x/y.png - x is in (
180°W ~ 180°E), y is in (85.0511°N ~ 85.0551°S), with the Y-axis going down from the top.
You can see a simple example of XYZ tiles from the Openlayers TileDebug Example.
TMS#
The TMS Wiki wikipedia does not cover many details, and the osgeo-specification only describes some application details of the protocol. Instead, the geoserver docs provide a more practical description of TMS. TMS is the predecessor of WMTS and is a standard established by OSGeo.
The request format is:
http://host-name/tms/1.0.0/layer-name/0/0/0.png
To support multiple file formats and spatial reference systems, multiple parameters can also be specified:
http://host-name/tms/1.0.0/layer-name@griset-id@format-extension/z/x/y
The standard TMS tile grid starts from the bottom left corner, with the Y-axis going up. Some map servers, such as geoserver, support an additional parameter flipY=true to flip the Y coordinate, making it compatible with service types where the Y-axis goes down from the top, such as WMTS and XYZ.
WMTS#
WMTS is more complex than the two intuitive protocols mentioned above and supports more scenarios. It was first published by OGC in 2010. Before this, after Allan Doyle's paper “Www mapping framework” in 1997, OGC began planning the establishment of standards related to web maps. Before WMTS, the earliest and most widely used web map service standard was WMS. Since each WMS request is organized based on the user's map zoom level and screen size, these responses vary in size. In the early days when multi-core CPUs were not so common, this on-demand real-time map generation method was very extravagant, and it was also very difficult to improve response speed. Thus, developers began to try pre-generating tiles, leading to many solutions, with TMS being one of them, and later WMTS emerged and began to be widely used. WMTS supports key-value pair (kvp) and RESTful methods for encoding request parameters.
KVP format:
<baseUrl>/layer=<full layer name>&style={style}&tilematrixset={TileMatrixSet}}&Service=WMTS&Request=GetTile&Version=1.0.0&Format=<imageFormat>&TileMatrix={TileMatrix}&TileCol={TileCol}&TileRow={TileRow}
RESTful format:
<baseUrl>/<full layer name>/{style}/{TileMatrixSet}/{TileMatrix}/{TileRow}/{TileCol}?format=<imageFormat>
Since it is raster tiles, we only need to find the correspondence between XYZ and the tile matrix and tile row/column numbers.
- TileMatrix
- TileRow
- TileCol
Here, the tile row and column numbers start from the top left, with the Y-axis going down.
Thus, we have found the correspondence between TMS and WMTS parameters, and the next step is how to convert TMS into WMTS requests, as follows:
- TileRow = 2^zoom - 1 - y = (1 << zoom) - 1 - y
- TileCol = x
- TileMatrix = zoom
Without considering other spatial references, the zoom level corresponds to the tile matrix, x corresponds to the tile column number, and y is inverted (because the starting direction is opposite).
Simulating a WMTS Capabilities Description File#
The requirements of the WMTS specification are almost detailed to the hair, so various clients, whether web-based Openlayers or desktop QGIS or Skyline, support directly parsing Capabilities description files and selecting layers, styles, and spatial references based on the content of the description file. Therefore, we also need to simulate a WMTS Capabilities description file.
Structure of the Capabilities Description File#
An example of a WMTS Capabilities description file can be found in the opengis schema and Tian Map Shandong.
The content of the Capabilities description file is very extensive; here are just some important parts (ignoring titles, contact information, etc.):
OperationsMetadata:
- GetCapabilities >> Method to obtain the Capabilities description file
- GetTile >> Method to obtain tiles
Contents:
- Layer
- boundingBox >> Longitude and latitude range of the layer
- Style
- TileMatrixSetLink >> Spatial reference supported by the layer
- TileMatrixSet >> Spatial reference
- TileMatrixSetLimits >> Zoom level range of the spatial reference
- TileMatrixLimits >> Tile row and column number range for each zoom level
- Style
- TileMatrixSet
- TileMatrix
The key parts are boundingBox, TileMatrixSetLimits, and TileMatrixLimits, which only need to be calculated based on the layer's spatial reference and zoom levels.
Calculating boundingBox is relatively simple, as it is the longitude and latitude range of the layer, so we won't elaborate on it here.
Calculating TileMatrixSetLimits is relatively straightforward, as it is the zoom level range of the layer's spatial reference.
Calculating TileMatrixLimits is more complex and can be done only when the layer range is relatively small; for global maps, it is unnecessary. It needs to be calculated based on the layer's spatial reference and zoom levels. Below is a piece of pseudocode (from 4326 to 3857).
FUNCTION GetTileRange(minLon, maxLon, minLat, maxLat, zoom, tile_size = 256)
minLonRad = minLon * PI / 180
maxLonRad = maxLon * PI / 180
minLatRad = minLat * PI / 180
maxLatRad = maxLat * PI / 180
tile_min_x = Floor((minLonRad + PI) / (2 * PI) * Pow(2, zoom))
tile_max_x = Floor((maxLonRad + PI) / (2 * PI) * Pow(2, zoom))
tile_min_y = Floor((PI - Log(Tan(minLatRad) + 1 / Cos(minLatRad))) / (2 * PI) * Pow(2, zoom))
tile_max_y = Floor((PI - Log(Tan(maxLatRad) + 1 / Cos(maxLatRad))) / (2 * PI) * Pow(2, zoom))
// adjust tile range based on tile size
tile_min_x = Floor((double)tile_min_x * tile_size / 256)
tile_max_x = Ceiling((double)tile_max_x * tile_size / 256)
tile_min_y = Floor((double)tile_min_y * tile_size / 256)
tile_max_y = Ceiling((double)tile_max_y * tile_size / 256)
RETURN (tile_min_x, tile_max_x, tile_min_y, tile_max_y)
Generating WMTS Capabilities Description File#
Generate a minimized WMTS Capabilities description file by filling in the key parts above, and then construct a RESTful style URL pointing to the standard description file address.
Conclusion#
The above is a simple idea of converting TMS to WMTS. In reality, there are many details to consider, such as spatial reference conversion, zoom level conversion, tile row and column number conversion, tile format conversion, etc. During this process, I also encountered some pitfalls, which I found more interesting.
In the first part, I quickly referred to the idea of tile-map-service-net5 and completed the conversion of y >> tileRow. The code is in WebMercator.cs. In fact, someone has asked this question on StackOverflow, and there is an answer, but I chose to find the answer from the software because it made me feel more secure.
The second part was quite challenging. First, I simulated the resource link and constructed a simple XML, but it could not be loaded directly on the target client. I immediately thought of testing through the standard service, and then I needed a Capabilities description file to modify. I initially wanted to test on Openlayers, which I was more familiar with, and then modify the Capabilities description file. Openlayers' loading method is still very flexible; without a Capabilities description file, it can directly access through configuration parameters.
// fetch the WMTS Capabilities parse to the capabilities
const options = optionsFromCapabilities(capabilities, {
layer: 'nurc:Pk50095',
matrixSet: 'EPSG:900913',
format: 'image/png',
style: 'default',
});
const wmts_layer = new TileLayer({
opacity: 1,
source: new WMTS(options),
})
Unfortunately, the tiles did not load, and there was not even a request sent in networks. So I went to another WMTS-related example, customized a TileGrid, and converted the tile row and column numbers to 3857's row and column numbers, and at that point, it could load.
const projection = getProjection('EPSG:3857');
const projectionExtent = projection.getExtent();
const size = getWidth(projectionExtent) / 256;
const resolutions = new Array(31);
const matrixIds = new Array(31);
for (let z = 0; z < 31; ++z) {
// generate resolutions and matrixIds arrays for this WMTS
resolutions[z] = size / Math.pow(2, z);
matrixIds[z] = `EPSG:900913:${z}`;
}
var wmtsTileGrid = new WMTSTileGrid({
origin: getTopLeft(projectionExtent), resolutions: resolutions, matrixIds: matrixIds,
})
After confirming that it was a TileGrid issue, I first compared my generated TileGrid with the TileGrid parsed from Capabilities by Openlayers. I found that some fields in my generated TileGrid were empty, so I tested each one, and finally discovered that when the internal parameters fullTileRanges_ and extent_ were empty, the images could load.
I went to check the OL source code and found that fullTileRanges_ and extent_ are used in getFullTileRange.
This means that when fullTileRanges_ and extent_ are empty, getFullTileRange will return an empty range.
And getFullTileRange is used in withinExtentAndZ, which is used to determine whether there are tiles of the layer in the current visible area. This means that when fullTileRanges_ and extent_ are empty, it cannot obtain TileRange, and withinExtentAndZ will always return true, thus continuously loading tiles, which is the reason for the successful loading.
Conversely, the fullTileRanges_ and extent_ parsed from Capabilities pointed to the wrong TileRange, causing withinExtentAndZ to always return false, thus preventing tiles from loading, which is the reason for the failure.
Finally, I found the reason, but I was misled again. In wmts.js, there is a comment in the constructor:
class WMTS extends TileImage {
/**
* @param {Options} options WMTS options.
*/
constructor(options) {
// TODO: add support for TileMatrixLimits
}
}
This made me initially mistakenly believe that fullTileRanges_ and extent_ were calculated based on the bounding box (boundingBox) rather than based on TileMatrixLimits. Therefore, I checked the bounding box again, confirmed it was correct, and then began to modify TileMatrixLimits.
At first, I thought TileMatrixLimits was the tile range for each level, rather than the layer's range, so I didn't pay attention to this parameter, which led to a detour.
Written in 2023, WMTS is no longer a new protocol, and the OGC Tile API has become a formal standard. I still have a superficial understanding of WMTS, and I feel quite embarrassed. 😅