Coinbase Exchange Inaccessible Due To 5x Traffic Spike During Bitcoin Surge

According to Coinbase, the exchange’s autoscaling couldn’t keep up with a 5x traffic spike experienced during a Bitcoin price surge to $10,000. Coinbase Exchange Inaccessible Due To 5x Traffic Spike During Bitcoin Surge

The largest cryptocurrency exchange in the United States said its autoscaling was unable to keep up with a huge traffic spike that left many users unable to log in on June 1.

According to a blog posted on June 6 by Coinbase software engineer Michael de Hoog, the exchange experienced a 5x traffic spike over four minutes around 16:05 PDT on June 1, around the time the price of Bitcoin (BTC) was approaching $10,000.

The software engineer said the exchange’s autoscaling was “unable to keep pace with this dramatic increase in traffic”:

“This traffic spike affected a number of our internal services, increasing latency between services. This led to process saturation of the web servers responsible for our API [application programming interface], where the number of incoming requests was greater than the number of listening processes, causing the requests to either be queued and timeout, or fail immediately. Our request error rate spiked to 50%, causing customers to experience errors when interacting with coinbase.com and our mobile apps.”

Coinbase said it redeployed the API at 16:20 to increase the number of machines dealing with this spike in traffic. Another two-minute outage followed “due to instances saturating and being marked unhealthy” before the exchange was back online.

History Of Outages

Cointelegraph reported last week that Coinbase has gone offline four times in the last three months during major Bitcoin price moves, leaving many users unable to access their portfolios.

However, Crypto Twitter later revealed the problems on the exchange went further back than that. CryptoWhale posted a chart to his 18,000+ followers on June 3 showing how Coinbase had gone offline 11 times in its history during major price moves:

After doing more research into exchanges uptimes, I’ve noticed a substandard pattern from #Coinbase.

Their exchange seems to be programmed to go “offline” anytime theres a $500+ move in #Bitcoin’s price. Over the last year, Coinbase has gone offline 11 times during larger moves. pic.twitter.com/3fNOU1QuiZ
— Whale ???? (@CryptoWhale) June 3, 2020

Unfortunately, outages during huge downturns or price surges hurt traders wanting to buy and sell the most. Losses can mount when Coinbase users aren’t able to access their accounts to sell their crypto, and potential profits can simply disappear when they can’t buy anything.

Will Coinbase Go Down Again?

The exchange said it was working on improvements in response to the June 1 outage. If Coinbase were to experience another traffic spike to the price of Bitcoin suddenly surging or falling, de Hoog said “pre-scaling and caching” would reduce the impact.

“Longer term we’re planning to improve our deployment process to mitigate some of the autoscaling issues we experienced,” the software engineer said.

Updated: 6-9-2020

Users Punish Coinbase For Outage by Withdrawing Record Amount of BTC

Coinbase experienced its highest single-day net BTC withdrawal in the wake of the outage incident, with even the author of “Black Swan” quitting the exchange.

Coinbase experienced the largest net outflow of Bitcoins (BTC) in ages in the wake of the recent service outage. Even Nassim Taleb has left the nest.

Black Swan

The June 1 outage happened during the surge of traffic as Bitcoin was trying to push through the $10,000 resistance level. Similarly, to the BitMex’ blackout a few months ago, this caused much frustration amongst its user base. A few days later, one of its most famous users, “Black Swan” author Nassim Taleb, quit the exchange.

Users Punishing Coinbase?

June 7, Coinbase users withdrew 22,000 more Bitcoins than they deposited — worth $214 million. According to Glassnode data, only once in history did Coinbase see a greater net withdrawal — July 28 2017, when the outflow was 22,500 BTC. The price at the time, however, was $2,785. Thus, denominated in dollars, that was a much more modest event, with only $63 million worth of Bitcoin withdrawn from the exchange.

Coinbase still holds close to a million Bitcoin, more than any other exchange. Thus, this latest incident may have an insignificant impact on its bottom line. Nonetheless, keeping in mind how BitMex lost its leadership position, Coinbase would be well-advised to fix its scalability woes in order not to alienate its users any further — black swans appear when you least expect them.

Updated: 6-9-2020

Coinbase Outlines Tech Plan To Help Avert Future Outages

After a traffic spike again caused severe disruption of its exchange service – and angered users – Coinbase said it’s working on technical changes aimed to provide more stability in the future.

In a blog post published late on Friday, Coinbase said it is ready to implement “a number of improvements,” which would allow the exchange’s servers to handle a sudden surge in usage – as was experienced last Monday.

“Around 16:05 PDT [23:05 UTC], the price of BTC reached USD $10,000. In connection with the rising price, we experienced a 5x traffic spike over 4 minutes. Our autoscaling was unable to keep pace with this dramatic increase in traffic,” reads the blog.

This traffic spike led to increased latency, Coinbase said, which had a domino effect on other services. With its servers becoming saturated with users trying to access and use the platform, and error reports also spiked by around 50% as visitors experienced timeouts and other server errors.

Within 20 or so minutes the firm had redeployed its API to increase the number of servers dealing with the traffic, as per the post.

After a review of the issues, Coinbase went on to list some of the changes that are currently being implemented to prevent similar traffic-spike outages from happening again:

The server “health check” system, which caused erroneous automated responses that worsened the June 1 issue, has already been updated to ensure that overloaded processes don’t get taken out of service.

Coinbase is also adjusting its systems to lower the impact of traffic spikes though “pre-scaling” – creating more server instances under heavy load. It will also rely more on caching, which has users load a stored version of pages in the browser, rather than reload full pages every time the website is visited.

“I personally think that autoscaling is great for the e-commerce industry” commented Paolo Ardoino, CTO at rival exchange Bitfinex. “Spikes in website traffic at exchanges can take place within milliseconds and place significant demands on an exchange’s capacity to maintain high availability at all times.”

Longer term, there are other changes planned too. Coinbase was light on the detail, however, saying it aims to improve its “deployment process to mitigate some of the autoscaling issues we experienced.

The exchange has something of a history of upsetting users with outages during important price moves. After online criticism reached conspiracy-theory levels after last week’s issue, Coinbase’s Justin Mart tweeted: “We do not purposely take down the site.”

CoinDesk reached out to Coinbase for more detail on the planned changes but hadn’t received a reply by press time.

Updated: 6-13-2020

Blackout Cure Needed? BTC Price Volatility Challenges Crypto Exchanges

2020 has been one of BTC’s most volatile periods, but why do cryptocurrency exchanges keep crashing under high demand?

Nobody can really predict Bitcoin’s (BTC) price volatility. But one thing is becoming painfully predictable when the price of Bitcoin suddenly lurches in one direction or another: One or more of the major cryptocurrency exchanges simply goes offline.

This leaves users powerless to prevent losses from spiraling, as they are left unable to trade or buy more positions as a hedge.

These outages have happened time and time again. Most recently, as Bitcoin started climbing toward the $10,000 mark, Coinbase went offline. At the time it happened, Cointelegraph reported that this is the fourth time in the last three months that Coinbase has shut down during major moves in the price of BTC. Additionally, Twitter user CryptoWhale pointed out that there have been no fewer than 11 Coinbase outages over the last 12 months, each one coming at a time when Bitcoin’s price had moved more than $500 in value.

The Silicon Valley-based exchange later issued a statement via its blog, clarifying that the June 3 outages were due to an issue with its API, which was seeing five times more traffic than usual. Without directly addressing the issue of the frequency of outages, the blog post stated that Coinbase is “working on reducing the impact of price-related traffic spikes though pre-scaling and caching.” Meanwhile, the exchange saw users withdrawing BTC en masse, following the incident.

A Broader Problem?

During March’s Black Thursday, BitMEX went offline for 25 minutes, subsequently blaming two separate distributed denial-of-service attacks. However, Twitter users, including the CEO of rival exchange FTX, Sam Bankman-Fried, and trader lowstrife, called foul play.

BitMEX denied the allegations, but it’s not the first time that the Seychelles-based exchange has been accused of playing dirty.

Blogger Hasu aired his suspicions that the company “weaponizes their server problems” back in 2018. This issue is also clearly set out in the form of allegations in the class-action lawsuit currently pending against BitMEX, which states: “BitMEX routinely freezing its servers — which BitMEX blames on technical glitches and limitations — to profit from moments of high volatility.”

A month after Black Thursday, the company saw a drop of 38% in its Bitcoin holdings. It is unclear whether the drop is due to users losing trust in the platform or because of the overall market sentiment, whereby an abnormally high amount of BTC is being withdrawn from exchanges. Meanwhile, BitMEX has struggled to regain the open interest lost in March, implying that it may be losing market share to its smaller rivals, such as Bybit and FTX.

Coinbase and BitMEX are the two platforms that have most often come under fire for downtime in volatile markets. However, data provider Kaiko performed an in-depth analysis of the minute-by-minute trade data for March 12 and 13, covering seven spot and six derivatives exchanges. Five of the spot venues and four of the derivatives platforms were found to have experienced some kind of issue during peak moments of volatility.

Of the spot platforms in question, only Binance and Bitstamp held up, although Binance CEO Changpeng Zhao acknowledged some “glitches on peripheral systems” in a tweet. In derivatives, Binance Futures and Huobi DM managed to ensure uninterrupted trading.

A Problem Unique To Crypto?

Contrary to the crypto sphere, there are no regular instances of the traditional stock markets going down during peak trading hours. Of course, the stock markets don’t see the same volatility as cryptocurrencies, but they do handle trading volumes that are far greater than any crypto exchange. The biggest cryptocurrency exchanges have enough experience of Bitcoin’s volatility to be able to anticipate certain kinds of peaks.

Arguably, some big exchanges also have money to invest in building infrastructure that can handle the kind of volume spikes. Coinbase has raised over half a billion dollars over its lifetime. It could be argued that BitMEX is a minnow, by comparison, having raised only $25,000 with the last seed round in 2015. However, one analyst estimates that BitMEX is raking in around $700,000 per day in fees from its derivatives trading service, which could come to over $250 million each year. Joel Edgerton, the chief operating officer of bitFlyer USA, believes that the issue is one of industry maturity, telling Cointelegraph:

“Crypto exchanges do not have the deep institutional experience that is in a traditional stock exchange. Traditional exchanges have had over 100 years to build the skills, processes and systems needed to handle the volumes they receive.”

Digging Deeper

Different exchanges appear to have different views on what could be an appropriate fix for the downtime problem. Bitfinex has recently issued a press release boasting its own performance in 2020, stating that it’s had no major incidents of downtime so far this year. The exchange points to its “obsessive interest in technical improvement” as being the reason why it has managed to achieve this.

Other exchange leaders seem to agree unanimously that the focus should be on the technology above all else. Edgerton explained that bitFlyer was built in Japan by specialists from Goldman Sachs and undergoes continual rigorous performance testing. On the matter, Catherine Coley, the CEO of Binance.US, pointed out to Cointelegraph that:

“Our infrastructure is built to regularly handle over $10 billion of daily trading activity. We have capacity for much larger volumes before our systems would become stressed.”

Stephen Stonberg, the COO of Bittrex, believes that many exchanges may be underestimating the effort and expertise involved in building an exchange that can withstand high volumes during peak times. He told Cointelegraph: “We can say with certainty that building a rock-stable exchange is harder than it looks.”

Unfounded Speculation?

While it seems acceptable enough that the crypto industry simply isn’t developed enough to handle high volatility at peak yet, the debate still rages over whether the industry should implement circuit breakers. Meanwhile, some have accused exchanges of using their downtime as a form of a hidden circuit breaker.

Several platforms that have suffered from outages, including Robinhood and Gemini, don’t offer leveraged trading.

Furthermore, both Coinbase and Gemini are regulated in the United States. Therefore, it would be a stretch to assume that the platforms take their services offline deliberately. If anything, spot exchanges risk losing out on the fees from users who would happily dump their holdings at market rates instead of setting a limit order.

Edgerton cautiously told Cointelegraph that there could be two reasons behind the outages: “In one theory, an exchange has lost focus on the cryptocurrency community and is trying to become a financial conglomerate with a hand in every pot.” He added: “In this case, frequent volatility-caused outages are the result of not taking care of the basics.” But it may also not just be about inadequate systems, as he elaborated on another potential theory:

“Some exchanges are closer to a casino business model than an exchange one. Volatile assets like Bitcoin do not need 100x leverage, especially when marketed to retail customers. Additionally, some of these exchanges may actively avoid regulation or not clarify where their business really exists. In these circumstances, it is reasonable that people would question mysterious outages as a hidden circuit breaker that is putting the company’s interests ahead of their customers.”

Users Demand Better Systems And Transparency

This year’s volatility has been extreme, even by Bitcoin’s standards. It’s also worth remembering that there are more traders, exchanges and general interest in cryptocurrencies now than at any point in the past. So, in a way, it’s no surprise that the infrastructure of some exchanges has started to creak and groan under heavier loads.

However, transparency is also critical. In traditional markets, regulation forces transparency. In crypto, only some exchanges have chosen to be regulated in order to operate in particular markets. In the absence of regulation, it’s reasonable to expect a certain degree of transparency from exchange operators, particularly regarding how they handle costly issues such as liquidation during volatility. In a crowded market, traders will vote with their feet, so it’s up to the crypto exchanges to meet transparency demands and system stability.

Updated: 1-29-2021

Coinbase Comes Back Online Following Issues

Coinbase users were reporting issues with certain purchases, though normal function has now seemingly returned.

Coinbase users recently reported suffering difficulties on the platform. Coinbase and its trading platform, Coinbase Pro, have a history of experiencing outages and other issues during high-traffic times.

Although users reported problems early this morning, a status update from Coinbase now indicates the platform is working normally once again. “All services are back up,” the update said at 12:33 pm EST. “We are currently monitoring our systems.”

“Coinbase has disabled buying and selling of some crypto assets,” Morgan Creek Digital co-founder Anthony Pompliano tweeted earlier on Friday morning. Multiple folks on Twitter posted pictures of their difficulties when attempting USD purchases on Coinbase.

The GameStop effect has completely upended the crypto market now. There’s a big Twitter-led $btc pump happening today and Coinbase — basically the Robinhood for entry-level crypto trading — has frozen USD purchases. Just tested it myself. pic.twitter.com/xpJCQjA0uh

— Ryan Broderick (@broderick) January 29, 2021

Crypto-Twitter personality Ryan Broderick also noted issues with buying BTC via the euro. He added in a separate tweet:

“For context, a bunch of big Twitter account have put #Bitcoin in their bios. Elon Musk being the biggest. Now $BTC is up almost 15% in the last 24 hours.”

Significant chatter has arisen in recent days surrounding drama in the financial markets and on social media. GameStop’s stock, under the ticker GME, enjoyed a significant rise in price, allegedly in connections with the efforts of a trading group on social media. Popular trading platform Robinhood recently suspended buying for a number of assets, including GME.

Coinbase often suffers platform downtime and technical difficulties. Multiple outage instances occurred in 2020, including during notable Bitcoin price moves in April and May.

Updated: 2-5-2021

Coinbase Outage Caused Crypto Purchase Issues, Not An Intentional Ban

Coinbase posted an in-depth explanation of its Jan. 29 platform difficulties.

On Jan. 29, a number of Coinbase users reported issues with the platform’s buy and sell functions. The Coinbase difficulties arose just a day after Robinhood barred GameStop share purchases following surging prices on the stock. A recent blog post indicates the Coinbase difficulties arose from an outage, not a ban on trading.

“Between 4:25am and 9:31am PST on Friday, January 29, api.coinbase.com had an outage,” Coinbase said in a post on Feb. 5.

“During this time, many users experienced errors while attempting to use the Coinbase app and crypto buying, selling and trading features were intermittently available,” the post added, also noting that users of its professional exchange, Coinbase Pro, were still able to sign on to the platform.

The cause of the problems on Coinbase? Crypto price data difficulties. “It was quickly diagnosed as an issue with a Redis cluster used to store spot rates for currency conversions,” the blog post detailed.

“These spot rates are used for estimated conversions between currencies, which puts it in the critical path for a range of functionality such as displaying the value of your portfolio in your local currency.”

Essentially, certain backend processes became overloaded and went down, based on subsequent information in the post. Coinbase’s tech team tried a number of options, eventually finding the right solution, which resurrected function on the platform.

The post also dove deeper, laying out a timeline of events on the backend during the technical disruption.

“Any requests which relied on exchange rate data were failing, and users were therefore unable to access critical functionality on our site,” the post noted as a result of the issues described in the timeline of backend events.

Coinbase listed multiple changes the team has implemented for improvement going forward. Platform difficulties are not foreign to Coinbase. The outfit has experienced many connectivity problems in the past. In November 2020, Brian Armstrong, Coinbase’s CEO, tweeted about the company’s work toward fixing its issues.

Updated: 5-25-2021

Upstart Peer-To-Peer Crypto Exchanges Take Aim At Coinbase

Decentralized markets such as Uniswap account for a growing share of cryptocurrency trading.

Coinbase Global Inc., COIN 7.59% which has made billions running cryptocurrency markets, is facing competition from a new breed of upstarts that look less like the New York Stock Exchange and more like Napster, the defunct music-sharing service.

Decentralized exchanges are peer-to-peer networks for swapping digital tokens. Last month, $122 billion in transactions took place on DEXes, as they are known, up from less than $1 billion a year earlier, according to data provider Messari.

Uniswap, the largest decentralized exchange, had volume of $36.6 billion in April, compared with $110 billion at Coinbase, Messari data show. Hayden Adams, a 27-year-old Brooklyn, N.Y., native, built the first version of Uniswap after being laid off from his job as an engineer at Siemens AG.

Unlike conventional crypto exchanges, DEXes don’t require users to hand their digital tokens to the exchange to be able to trade. That appeals to traders worried about losing their holdings to hackers who have a long history of targeting crypto exchanges for theft.

There is no central authority on a decentralized exchange to decide who is allowed to trade, or what tokens can be traded. Uniswap can be used to trade more than 30,000 unique tokens.

“It’s like eBay. If you want to sell something, you can list it,” said Boris Wertz, founding partner of Version One Ventures, a venture-capital firm that invested in Uniswap last year alongside Andreessen Horowitz, one of Silicon Valley’s most prominent VC firms.

Decentralized exchanges hark back to the initial vision of cryptocurrencies as a new type of financial system, free from middlemen. But some lawyers warn that DEXes could be on a collision course with regulators.

DEXes generally don’t have safeguards against money laundering, or “know your customer” measures in which an exchange confirms the identity of traders using the platform. That could raise red flags for government authorities, especially as DEXes gain popularity, said Ashley Ebersole, a former Securities and Exchange Commission attorney.

“If a lot of money is moving across your platform, regulators could have serious problems if you’re not asking even basic questions to know who’s involved in those transactions and where they’re located,” said Mr. Ebersole, who is now a partner at Bryan Cave Leighton Paisner LLP.

There is evidence of criminals using decentralized exchanges. After the September theft of $281 million from KuCoin, an overseas crypto exchange, the thieves used Uniswap to trade $10.5 million of stolen coins for ether, a key step in their effort to launder the proceeds of the heist, according to Elliptic, a blockchain-analytics firm.

Another analytics firm, Chainalysis, has linked the KuCoin theft to a criminal syndicate working on behalf of the North Korean government.

Mr. Adams, the creator of Uniswap, said in an interview that he was “no fan” of hackers or scammers. But he stressed that Uniswap was simply a protocol—a way for computers to talk to each other—that could be used for good or evil.

“It is like the internet,” he said. “There are good things that happen on the internet, and there are bad things that happen on the internet. The internet is this neutral infrastructure.”

Moreover, because there isn’t a central set of servers that runs Uniswap, it can’t be shut down. It is also unclear how the government could require any entity to act as a gatekeeper and identity people using the protocol. Mr. Adams leads Uniswap Labs, a firm that has developed much of Uniswap, but his team has handed governance of the protocol to a broad community of users that can vote on policy changes.

Community members gain voting power based on their holdings of UNI, a digital token that was distributed by Uniswap’s team last year. The total, fully diluted value of those tokens is about $24 billion, a number sometimes cited as Uniswap’s valuation.

About 21% of UNI tokens were set aside for the Uniswap team, while an additional 18% were designated for investors. That means Mr. Adams and other current and future Uniswap Labs employees own a digital stash worth more than $5 billion at today’s prices, although they can’t tap it yet, due to a four-year vesting schedule.

Several years ago, Mr. Adams was jobless and living in his father’s house outside New York. Having recently gotten into crypto, he started a project: building a decentralized exchange based on the Ethereum blockchain network.

Early DEXes struggled to provide good prices for their users because they didn’t have a way to attract market makers—firms that continually buy and sell financial assets, quoting prices that other market participants can trade against.

Mr. Adams’s breakthrough, inspired by a blog post by Ethereum founder Vitalik Buterin, was to design a system of “automated market makers.” These are computer programs that build up pools of tokens contributed by Uniswap users and determine prices using a mathematical formula, based on the supply of tokens in the pools.

In exchange for posting tokens into these pools, Uniswap users get a small percentage of the value of every trade, effectively letting them earn a yield for their crypto assets. More than 100,000 users are now acting as such “liquidity providers” on Uniswap.

It is unclear how much of a threat DEXes pose to centralized exchanges. One major limitation is that they can be used only for crypto-to-crypto trading, so investors seeking to swap dollars for digital currencies must use Coinbase or other conventional exchanges.

Still, the incumbents have taken note. Coinbase listed competition from decentralized platforms as a potential business risk before going public last month.

Mr. Adams predicted that major exchanges would eventually tap into Uniswap’s liquidity pools, essentially becoming front-end interfaces with slick apps and websites that rely on Uniswap behind the scenes to power trading.

To some extent, that is already happening. Coinbase’s wallet service, which lets people hold crypto assets, connects to Uniswap and other DEXes, allowing customers to buy coins unavailable on Coinbase’s exchange.

There is room for both centralized and decentralized exchanges to grow, according to Coinbase Chief Product Officer Surojit Chatterjee. “Crypto isn’t a zero-sum game,” he said. “We see DEXes as a way to expand opportunities for our users.”

Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,Coinbase Exchange Inaccessible Due,

Bitcoin Information & Resources (#GotBitcoin?)

US Pentagon Created A War Game To Fight The Establishment With BTC (#GotBitcoin?)

Family Offices Finally Accept The Benefits of Investing In Bitcoin

An Army Of Bitcoin Devs Is Battle-Testing Upgrades To Privacy And Scaling

Bitcoin ‘Carry Trade’ Can Net Annual Gains With Little Risk, Says PlanB

Max Keiser: Bitcoin’s ‘Self-Settlement’ Is A Revolution Against Dollar

Blockchain Can And Will Replace The IRS

China Seizes The Blockchain Opportunity. How Should The US Respond? (#GotBitcoin?)

Jack Dorsey: You Can Buy A Fraction Of Berkshire Stock Or ‘Stack Sats’

Bitcoin Price Skyrockets $500 In Minutes As Bakkt BTC Contracts Hit Highs

Bitcoin’s Irreversibility Challenges International Private Law: Legal Scholar

Bitcoin Has Already Reached 40% Of Average Fiat Currency Lifespan