Skip to content

Optimize preferredProduct fdsnws view, Fix getTsunamiLinkProduct

While investigating how tsunami links are summarized, I noticed the preferredProduct view was extremely slow for the Ridgecrest event. This event has 1600+ products and has revealed other performance issues.

getTsunamiLinkProduct also considers old products that have since been deleted.

Existing query using not exists:

MySQL [indexer]> select * from preferredProduct where eventid=2877
    -> ;
+---------+----------+----------------+
| eventId | id       | type           |
+---------+----------+----------------+
|    2877 | 19367810 | dyfi           |
|    2877 |  9490561 | finite-fault   |
|    2877 | 11130612 | general-link   |
|    2877 |  9495156 | general-text   |
|    2877 |  9859436 | ground-failure |
|    2877 | 13376425 | impact-text    |
|    2877 | 12028560 | losspager      |
|    2877 | 12390424 | moment-tensor  |
|    2877 | 11611132 | nearby-cities  |
|    2877 | 19430721 | oaf            |
|    2877 |  1965197 | origin         |
|    2877 |  9977051 | phase-data     |
|    2877 |  9701999 | shakemap       |
+---------+----------+----------------+
13 rows in set (9 min 33.01 sec)

Proposed query using left join instead of not exists:

MySQL [indexer]> SELECT ps.eventId, ps.id, ps.type
    ->   FROM (
    ->    SELECT *
    ->    FROM currentProducts
    ->    WHERE eventId IS NOT NULL
    ->    AND status <> 'DELETE'
    ->   ) ps
    ->   LEFT JOIN (
    ->     SELECT *
    ->     FROM currentProducts
    ->     WHERE eventId IS NOT NULL
    ->     AND status <> 'DELETE'
    ->   ) morePreferred
    ->   ON (
    ->     morePreferred.eventId = ps.eventId
    ->     AND morePreferred.type = ps.type
    ->     AND (
    ->       morePreferred.preferred > ps.preferred
    ->       OR (
    ->         morePreferred.preferred = ps.preferred
    ->         AND morePreferred.updateTime > ps.updateTime
    ->       )
    ->     )
    ->   )
    ->   WHERE morePreferred.id IS NULL
    ->   AND ps.eventId = 2877;
+---------+----------+----------------+
| eventId | id       | type           |
+---------+----------+----------------+
|    2877 | 19367810 | dyfi           |
|    2877 |  9490561 | finite-fault   |
|    2877 | 11130612 | general-link   |
|    2877 |  9495156 | general-text   |
|    2877 |  9859436 | ground-failure |
|    2877 | 13376425 | impact-text    |
|    2877 | 12028560 | losspager      |
|    2877 | 12390424 | moment-tensor  |
|    2877 | 11611132 | nearby-cities  |
|    2877 | 19430721 | oaf            |
|    2877 |  1965197 | origin         |
|    2877 |  9977051 | phase-data     |
|    2877 |  9701999 | shakemap       |
+---------+----------+----------------+
13 rows in set (0.57 sec)

MySQL [indexer]>
Edited by Jeremy M Fee

Merge request reports

Loading