tag:pugpig.statuspage.io,2005:/historyPugpig Status - Incident History2024-03-29T05:22:53ZPugpigtag:pugpig.statuspage.io,2005:Incident/202673722024-03-16T10:15:56Z2024-03-16T10:15:56ZExpress CMS Scheduled Upgrades<p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>10:15</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>13:19</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing scheduled upgrades at this time. There will be delays to the publishing queues, but we do not anticipate the actual downtime to be longer than 5 minutes.</p>tag:pugpig.statuspage.io,2005:Incident/200782752024-02-29T12:00:57Z2024-02-29T12:00:57ZAzure storage migration (Pugpig Archive/Bondi only)<p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>12:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>09:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>26</var>, <var data-var='time'>12:29</var> UTC</small><br><strong>Scheduled</strong> - We are migrating our Azure classic storage to the newer resource manager.<br /><br />We expect minimal rolling per-customer downtime as we migrate each customer in turn.</p>tag:pugpig.statuspage.io,2005:Incident/200782722024-02-28T12:00:58Z2024-02-28T12:00:58ZAzure storage migration (Pugpig Archive/Bondi only)<p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>12:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>09:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>26</var>, <var data-var='time'>12:28</var> UTC</small><br><strong>Scheduled</strong> - We are migrating our Azure classic storage to the newer resource manager.<br /><br />We expect minimal rolling per-customer downtime as we migrate each customer in turn.</p>tag:pugpig.statuspage.io,2005:Incident/200782632024-02-27T11:24:04Z2024-02-27T11:25:07ZAzure storage migration<p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>11:24</var> UTC</small><br><strong>Completed</strong> - We have completed all the customer migrations for today.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>09:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>26</var>, <var data-var='time'>12:27</var> UTC</small><br><strong>Scheduled</strong> - We are migrating our Azure classic storage to the newer resource manager.<br /><br />We expect minimal rolling per-customer downtime as we migrate each customer in turn.</p>tag:pugpig.statuspage.io,2005:Incident/200504092024-02-24T11:00:56Z2024-02-24T11:00:56ZScheduled Maintenance needed<p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>11:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>22</var>, <var data-var='time'>18:07</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance at this time. We don't expect the downtime in the course of the course to be more than 10 minutes.</p>tag:pugpig.statuspage.io,2005:Incident/194283422023-12-08T18:00:00Z2023-12-13T14:50:42ZDistribution content ingestion from Express failures<p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>18:00</var> UTC</small><br><strong>Resolved</strong> - Caching issues relating to network activation of a plugin stopped distribution from being able to process content from Express site until each site's admin page was visited at least once.</p>tag:pugpig.statuspage.io,2005:Incident/194282882023-12-06T15:00:00Z2023-12-13T14:33:48ZExpress import performance degraded<p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>15:00</var> UTC</small><br><strong>Resolved</strong> - Due to unexpected load caused by 1M archive pages being imported in one go, we suffered unusually high queuing delays to imports.</p>tag:pugpig.statuspage.io,2005:Incident/189982662023-11-04T12:27:43Z2023-11-04T12:27:43ZEC2 instance reboot maintenance scheduled by AWS for Pugpig Pro<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>12:27</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance is now complete.<br /><br />Operator error during the migration inadvertently overrode our automatic DNS updating infrastructure, thereby delaying the propagation of the correct new IP address of one of the instances slightly.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>12:01</var> UTC</small><br><strong>Update</strong> - We are continuing to verify the maintenance items.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:56</var> UTC</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'> 2</var>, <var data-var='time'>09:31</var> UTC</small><br><strong>Scheduled</strong> - As part of AWS' routine fleet management they require us occasionally to restart and/or move instances around.<br /><br />We expect brief downtime of some of our Pugpig Pro servers during this window. Note this does not impact Express customers at all.</p>tag:pugpig.statuspage.io,2005:Incident/187229112023-10-08T18:04:08Z2023-10-08T18:04:08ZCertain users unable to remain logged in to the Distribution Service<p><small>Oct <var data-var='date'> 8</var>, <var data-var='time'>18:04</var> UTC</small><br><strong>Resolved</strong> - This has now been resolved.</p><p><small>Oct <var data-var='date'> 8</var>, <var data-var='time'>11:02</var> UTC</small><br><strong>Identified</strong> - We have identified the root cause - there is an AWS session replication problem across our load balancers. If you select "Remember Me" when logging in, you should avoid the problem.</p><p><small>Oct <var data-var='date'> 8</var>, <var data-var='time'>09:33</var> UTC</small><br><strong>Investigating</strong> - We are investigating an issue where certain users are unable to remain logged into the web UI of the Distribution Service.<br /><br />Note that this does not affect content processing, the Distribution API or the Distribution Content Widget.</p>tag:pugpig.statuspage.io,2005:Incident/186896152023-10-07T11:00:25Z2023-10-07T11:00:25ZUpgrade to Distribution AWS Components<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>11:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>10:26</var> UTC</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>10:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'> 4</var>, <var data-var='time'>15:25</var> UTC</small><br><strong>Scheduled</strong> - We recently suffered two incidents relating to a rate-limit imposed by one of our AWS Services. We will be applying our mitigation to our Production Servers on Saturday to ensure this cannot happen again. We do not expect any disruptions or downtime, but we are reserving this window just in case.</p>tag:pugpig.statuspage.io,2005:Incident/186777582023-10-03T16:20:12Z2023-10-03T16:20:12ZPugpig Distribution Service Down<p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>16:20</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>13:30</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>13:28</var> UTC</small><br><strong>Monitoring</strong> - The service is restored. We are monitoringit.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>13:05</var> UTC</small><br><strong>Identified</strong> - We have once again hit a rate limit with an AWS Service that has stopped the Distribution Service. We have identified the issue and are working on restoring it.</p>tag:pugpig.statuspage.io,2005:Incident/186676772023-10-02T10:10:00Z2023-10-06T13:46:17ZPugpig Distribution Service Down<p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>10:10</var> UTC</small><br><strong>Resolved</strong> - This morning from 9:11 UTC to about 10:10 UTC, the Pugpig Distribution Service failed following a deployment of an update.<br /><br />After the failure to start, our engineers investigated, and discovered this was because we were hitting a rate limit on one of our AWS Services (specifically the AWS SSM Parameter Store). This was most likely a chance alignment of an unusually busy set of deployments for multiple environments which lead to automatic re-starts making it more likely to hit the rate limit.<br /><br />In order to restore the service, we forcibly stopped all automatic re-starts, then re-started the distribution production service manually.<br /><br />During the outage, the Distribution Service was unavailable, and content updates would have stalled.<br /><br />We're currently implementing improved designs to avoid this issue in the future.</p>tag:pugpig.statuspage.io,2005:Incident/184204542023-09-09T10:30:21Z2023-09-09T10:30:22ZUpdate to Pugpig AWS Databases<p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>10:30</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'> 7</var>, <var data-var='time'>11:01</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing scheduled database upgrades at this time. We expect the downtime to be less than 5 minutes but it may be as long as 20. Both Express and Distribution will not be available during the window. Content processing will pause during the window.<br /><br />The apps will continue to operate as normal during the window.</p>tag:pugpig.statuspage.io,2005:Incident/175201172023-06-09T09:07:56Z2023-06-09T09:07:56ZExpress CMS performance degraded<p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>09:07</var> UTC</small><br><strong>Resolved</strong> - We have identified the root cause of this problem. Some of our clients use a third party plugin called Advanced Taxonomy Term Order. This is a popular plugin - we have a licence for it. However, we've learned that the plugin does a licence check with the servers at http://www.nsp-code.com/ - which are currently down, and this was blocking/slowing down requests.<br /><br />We have put in a workaround to stop this check, and are looking into a permanent fix to ensure this cannot happen again.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>08:25</var> UTC</small><br><strong>Monitoring</strong> - We have identified a slow component and have increased capacity. Services should be returning to normal. This component only affected a small number of customers with a specific kind of custom importer.<br /><br />We are continuing to monitor.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>07:32</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:pugpig.statuspage.io,2005:Incident/167713892023-04-06T21:12:12Z2023-04-07T19:57:54ZArchive Sites down due to Azure issues in East US Region<p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>21:12</var> UTC</small><br><strong>Resolved</strong> - Microsoft has confirmed that this is resolved. Apologies for the service interruption.</p><p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>20:30</var> UTC</small><br><strong>Monitoring</strong> - The archive sites all appear to have returned to normal, so it seems Microsoft have fixed things. However, we are still waiting to hear more from their support team so will continue to monitor them.</p><p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>18:36</var> UTC</small><br><strong>Update</strong> - We have received the following update from Microsoft Support:<br /><br />STATUS:<br />In-Progress 4/6/2023, 1:30:06 PM UTC<br /><br />SUMMARY OF IMPACT:<br />Impact Statement: Starting at 11:50 UTC on 06 Apr 2023, you have been identified as a customer using App Service in East US or North Central US who may receive intermittent HTTP 500-level response codes, experience timeouts or high latency when accessing App Service (Web, Mobile, and API Apps), App Service (Linux), or Function deployments hosted in this region.<br /><br />Current Status: Our investigation suggests that a recent deployment task on a backend instance that App Service utilizes became unhealthy, causing users to experience the issues mentioned above. The root cause has been identified, mitigation strategies are currently being evaluated and the fix is currently being prepared. Next update will be provided in 60 minutes, or as events warrant.<br /><br />NEXT STEPS:<br />We are urgently investigating the root cause and coming up with a plan to resolve as soon a possible. We'll send an update once we know more.</p><p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>18:24</var> UTC</small><br><strong>Identified</strong> - We are expecting an update from Microsoft Support on this in the next hour.</p><p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>16:35</var> UTC</small><br><strong>Update</strong> - Microsoft are still investigating. There seem to be a few different issues happening. At present, about half of the Pugpig Archive sites are fine (the ones in a different region). Of the broken ones, half are displaying a "The page cannot be displayed because an internal server error has occurred." fatal error, while the others are always displaying the Page Not Found page.<br /><br />Other users are also complaining about this outage on Twitter, although the Microsoft Azure status page (https://azure.status.microsoft/) does not have any info as of now.</p><p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>15:52</var> UTC</small><br><strong>Investigating</strong> - Many of our Archive sites are currently down or returning errors due to a Microsft Azure East US issue . We currently have an open ticket with Microsoft. The latest update:<br /><br />"As we had discussed on this call, I looked into the service described in this support ticket for any potential issues from the application. At this time, the pattern I am noticing is the same as others in this region. Our engineers are currently investigating the cause of this issue and are working to mitigate this as soon as possible."</p>tag:pugpig.statuspage.io,2005:Incident/166424372023-03-26T11:15:17Z2023-03-26T11:15:17ZExpress CMS Maintanance<p><small>Mar <var data-var='date'>26</var>, <var data-var='time'>11:15</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>26</var>, <var data-var='time'>11:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>25</var>, <var data-var='time'>13:02</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:pugpig.statuspage.io,2005:Incident/159305742023-01-25T11:15:30Z2023-01-25T11:15:30ZUnplanned Maintanance to Express CMS<p><small>Jan <var data-var='date'>25</var>, <var data-var='time'>11:15</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'>25</var>, <var data-var='time'>11:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'>25</var>, <var data-var='time'>10:43</var> UTC</small><br><strong>Scheduled</strong> - We need to do an emergency upgrade to the Express servers. We expect the CMS to be down for about 5 minutes, but it might be slightly longer.</p>tag:pugpig.statuspage.io,2005:Incident/127758942022-11-12T12:00:36Z2022-11-12T12:00:36ZUpgrade to Pugpig Express Databases<p><small>Nov <var data-var='date'>12</var>, <var data-var='time'>12:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Nov <var data-var='date'>12</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>10</var>, <var data-var='time'>12:37</var> UTC</small><br><strong>Scheduled</strong> - We will be upgrading the Pugpig Express Databases. During this time, the CMS will not be available, and content updates will not flow through the system.</p>tag:pugpig.statuspage.io,2005:Incident/126967532022-11-05T11:30:35Z2022-11-05T11:30:35ZUpgrade to Pugpig Pro and Pugpig Site Databases<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>11:30</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>10:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'> 2</var>, <var data-var='time'>17:21</var> UTC</small><br><strong>Scheduled</strong> - We are upgrading our databases for our Pro and Site clients. During this time, the CMS will not be available, and content feeds will not be processed.<br /><br />The Pro apps and Sites should continue to work with the content they had. Sites should be unaffected unless the CDN ejects pages from their cache, in which case the page will only return once the maintenance is complete.<br /><br />Express clients are not affected by this upgrade.</p>tag:pugpig.statuspage.io,2005:Incident/123772582022-10-29T11:00:29Z2022-10-29T11:00:29ZUpgrade to Distribution Service<p><small>Oct <var data-var='date'>29</var>, <var data-var='time'>11:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>29</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>20</var>, <var data-var='time'>15:01</var> UTC</small><br><strong>Scheduled</strong> - We are upgrading our systems to improve performance. Distribution will be unavailable for approximately one hour during this time, and content will not update. The apps will continue to function as normal.</p>tag:pugpig.statuspage.io,2005:Incident/121183992022-10-16T10:22:59Z2022-10-16T10:22:59ZSearch and Authentication APIs are returning slowly or timing out<p><small>Oct <var data-var='date'>16</var>, <var data-var='time'>10:22</var> UTC</small><br><strong>Resolved</strong> - We have discovered an underlying network issue that was causing various timeouts. This has been rectified and things appear better. We will continue to monitor.</p><p><small>Oct <var data-var='date'>16</var>, <var data-var='time'>08:17</var> UTC</small><br><strong>Update</strong> - The problem seems to be mainly caused for clients that are using Piano as their authentication system. We are continuing to monitor.</p><p><small>Oct <var data-var='date'>16</var>, <var data-var='time'>07:48</var> UTC</small><br><strong>Monitoring</strong> - We are continuing to monitor this issue - performance is improving and errors should be less likely at the moment.</p><p><small>Oct <var data-var='date'>16</var>, <var data-var='time'>07:06</var> UTC</small><br><strong>Investigating</strong> - Search and Authentication APIs are returning slowly or timing out. This may cause users to be unable to log in, or see errors while searching. Users who are already logged in should not be affected.</p>tag:pugpig.statuspage.io,2005:Incident/107446872022-08-03T13:43:35Z2022-08-03T13:43:35ZPro & Site: Intermittent database issues<p><small>Aug <var data-var='date'> 3</var>, <var data-var='time'>13:43</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>20:53</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>20:36</var> UTC</small><br><strong>Update</strong> - All systems have returned to normal, but we are continuing to monitor. One of the instances in our Pro/Site server fleet was creating far too many database connections, blocking connections for others. We have now applied a maximum number of connections per server to prevent this from reoccurring.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>17:36</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>16:21</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>15:37</var> UTC</small><br><strong>Update</strong> - We are still investigating the root cause of this issue. Errors are still happening intermittently. Note that live apps are unaffected, although updates for live apps running on a Pugpig Pro CMS may be slower than normal.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>14:47</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Aug <var data-var='date'> 2</var>, <var data-var='time'>14:47</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating an intermittent issue with Pro & Site CMSs - primarily affecting CMS admin access. We will update again soon.</p>tag:pugpig.statuspage.io,2005:Incident/100632542022-05-24T11:53:36Z2022-05-24T11:53:36ZIssues with Distribution Databases<p><small>May <var data-var='date'>24</var>, <var data-var='time'>11:53</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>24</var>, <var data-var='time'>11:22</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>May <var data-var='date'>24</var>, <var data-var='time'>11:08</var> UTC</small><br><strong>Investigating</strong> - We are currently have an issue with the Amazon AWS database behind the Distribution Service. Users will be unable to access the web interface, and some processing may be delayed. We are currently investigating.</p>tag:pugpig.statuspage.io,2005:Incident/99678212022-05-11T17:23:58Z2022-05-11T17:23:58ZDatabase Access issues for certain Pugpig Site and Pugpig Pro customers<p><small>May <var data-var='date'>11</var>, <var data-var='time'>17:23</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>11</var>, <var data-var='time'>16:32</var> UTC</small><br><strong>Monitoring</strong> - It appears an AWS RDS autoscaling task took longer than normal. This is now complete and operations appear normal again.<br /><br />We will continue to monitor it.</p><p><small>May <var data-var='date'>11</var>, <var data-var='time'>16:20</var> UTC</small><br><strong>Investigating</strong> - We are seeing elevated database errors for certain CMS users trying to access their Pugpig Site or Pugpig Pro CMS.<br /><br />We are investigating the issue.</p>tag:pugpig.statuspage.io,2005:Incident/98778362022-04-29T10:26:13Z2022-04-29T10:26:13ZIssues accessing the Distribution Web Interface<p><small>Apr <var data-var='date'>29</var>, <var data-var='time'>10:26</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>29</var>, <var data-var='time'>08:24</var> UTC</small><br><strong>Monitoring</strong> - Service has been restored. We are monitoring and looking into the root cause.</p><p><small>Apr <var data-var='date'>29</var>, <var data-var='time'>08:13</var> UTC</small><br><strong>Investigating</strong> - Users are currently unable to access the Distribution Service web interface. Content processing does not seem to be affected. We are investigating the issue.</p>