The following article has been machine translated, and may not be completely accurate. If you'd like to view the original interview in Japanese, click here.
Cybozu has been providing services and software to support business operations, mainly through its cloud-based groupware service 'cybozu.com' and business application platform 'kintone'. Cybozu continues to create collaboration tools based on its philosophy of "creating a society where teamwork thrives", and has more than 85,000 companies using its products.
For Cybozu, the security of its services and products is an essential element. Due to the nature of the information handled by Cybozu products, we believe it is essential to protect not only confidentiality but also availability and integrity," says Yuriko Otsuka of Cy-PSIRT.
Yuriko Otsuka, Cybozu Cy-PSIRT
Against this background, Cybozu was the first in the industry to establish a Product Security Incident Response Team (PSIRT) dedicated to improving the security quality of its services and products, and has been working to create products with fewer vulnerabilities.
Full confidence in Gehirn's technical competence
It was only recently, when the risk of cyber-attacks had been recognised, that more and more companies have set up CSIRTs (Computer Security Incident Response Teams) to protect the security of the whole company and respond to incidents.
In the case of Cybozu, however, the order is a little different. First, we established a security team as part of quality assurance (QA) in the Development Division, which is responsible for product development, and it developed into PSIRT in 2011. In this process, the PSIRT started to receive a wide range of security-related enquiries, so we separated the general security support from the CSIRT and set up the CSIRT so that the PSIRT can concentrate on the security of the product, which is its main task.
Ms. Otsuka says, "We started to secure our products before we set security rules for the company." The reason for this unique situation is that we once had an experience where an external expert pointed out a vulnerability in one of our products. After this incident, Cybozu began to take the security of its products seriously, and strengthened security inspections at the development and testing stages, and created a process to identify and address problems at an early stage before release. This is known as "Secure by Design" or "Shift Left".
Nevertheless, before the establishment of PSIRT, each tester in charge of a product was responsible for verification. As a result, the level of verification varied depending on the knowledge of the testers about security, and sometimes omissions occurred.
On the other hand, after the PSIRT started to operate in earnest, the verification process has been changed to a company-wide verification process based on uniform rules. In addition to this, we commissioned a US company to conduct a security inspection by external eyes, and also commissioned an external company to conduct a "security audit" once or twice a year, the results of which are published in an audit report.
Cybozu hired Gehirn to perform this security audit. After getting acquainted with Gehirn through the connection of the then director of Gehirn, they asked Gehirn to do an inspection once and trusted Gehirn's high technological prowess, and the number of requests naturally increased.
"The report was of a very high standard, as were the results. We continued to ask Gehirn for further audits, and each time they pointed out different things at a high level, which built up our trust," says Otsuka. Based on these results, in 2017, the company has commissioned Gehirn to conduct most of its audits and has made the results of these reports available to the outside world (https://www.cybozu.com/jp/productsecurity/).
A series of internal and external inspections, as well as audits by Gehirn, reduce blind spots.
Cybozu tests the security of its products not just once, but two or three times. "It's true that testing in three stages is expensive. But each has a different purpose and coverage, so it's not something that can be covered by just one," says Otsuka.
"Internal security checks are comprehensive and have the advantage that they are done internally, so the specifications can be checked, and they are more likely to point out issues such as privilege escalation. External inspections, which we commission from US companies, tend to point out things like the use of outdated open source software, in addition to the more benign specifications, where Gehirn has pointed out a wide and deep range of issues. Each inspection has its own strengths and layers, and they make the most of these checks.", said Otsuka.
Of course, there are areas of overlap between the three types of tests. But there are also areas that are thin, and we have a healthy amount of concern that there may be vulnerabilities in those areas. "We want to spend more money on those areas," says Otsuka. On top of that, the company also has a "vulnerability bounty system" to receive vulnerability information from security experts in the general public, and is trying to reduce blind spots by paying more attention to them.
This is a bit of a pressure for Gehirn, the person who conducts the audit. Gehirn's Hirasawa says, "To be honest, the security of Cybozu's products is very solid, and we hardly find any basic vulnerabilities compared to other companies. Especially for new products, I think it's because Cybozu is so conscious of security from the beginning of development. It's hard for us to write a report saying 'no vulnerabilities', and as an engineer it's frustrating, but it's true that we feel it's difficult to find vulnerabilities."
Ren Hirasawa, Gehirn Technical Analysis Division
However, in general, more and more applications are being developed these days relying on frameworks. For this reason, "the places where vulnerabilities occur are changing depending on the language and frameworks used. Some, such as Ghostscript and ImageMagick vulnerabilities, often contain critical vulnerabilities in components that developers are unaware that they are using. In some cases, these vulnerabilities remain for months after they are discovered because the developer is unaware of the dependency in the first place.", says Hirasawa.
Cybozu has also started to recognize these issues. Ms. Otsuka says, "In order to keep track of what external libraries, middleware and open source software we rely on, we ask our customers to provide us with information on the components they use in their products, which we manage centrally in PSIRT, and we manually check for updates on a weekly basis." However, he says that the volume of these updates is so large that it is difficult to keep up with them.
Easy-to-understand reporting encourages development to make changes
Another point that Cybozu appreciates about Gehirn's diagnostic service is the clarity of the report. Gehirn's president, Daiki Ishimori, has always focused on making reports so that developers can understand the intentions of the reports when they read them. The format of the overall evaluation has been improved through trial and error, so that the reader can easily understand whether the situation is dangerous or not.
"PSIRT activities do not end with the detection of vulnerabilities, but through communicating the information, together with the risks, to the product developers, and only after the modifications are completed can the product be made robust. However, developers and product managers are not necessarily security experts. I think it is important to communicate the risks in a way that is easy for them to understand, and the reports provided by Gehirn can be used for this purpose.", says Otsuka.
At Cybozu, security inspections are conducted at multiple steps, and PSIRT gives advice based on the results, but the decision on when to fix the problems pointed out in the inspections lies with the development side. Therefore, if the risks are not communicated properly, errors in judgement can occur, and even if there is no ill will, corrections can be postponed until after the other side has been dealt with. Of course, if you are aware of the risks, you may decide to delay the revision. There is only so much that can be done with a limited amount of manpower, so the priority should be determined based on the defects rather than vulnerabilities.
"There are some issues that have a low CVSS score, but where the risk is too great to be exposed. We communicate with development in a variety of ways, and we are grateful for this detailed report as it helps to persuade them.", says Otsuka.
Another thing that Cybozu finds very useful is the "Reference Information". This is a summary of new features that are being introduced in some parts of the world, such as the SameSite attribute of cookies, and items that are not yet a risk but may become a risk in the future depending on the situation and how they are used, such as "inadequate escaping" that cannot be abused by itself. This is a summary. This is also "very helpful", says Ms Otsuka.
A continuous and passionate exchange of information to increase the number of colleagues involved in security.
Since we started using Gehirn's security testing service - "We have had no problems or complaints. We are very satisfied with the content of the detection and the cost is also very reasonable. Above all, I can feel the trustworthy personality of the people at Gehirn in the communication before and after the diagnosis, which is very helpful," said Otsuka.
As a security engineer, Ms. Otsuka felt sympathy with an item on Gehirn's recruitment page that says "People who have ever looked for vulnerabilities in their dreams". From the words and expressions on his face, we could tell that he liked technology and that he was doing what he loved.
Recently, PSIRTs have started to show their trust in Gehirn, saying that "Gehirn finds things that we can't find on our own", and when they say "This version will be audited by Gehirn", they sometimes get the response "Good, that's a relief". The company plans to expand the number of products to be audited, with nearly 20 audits in 2019.
In the future, Cybozu plans to assign at least one engineer with strong security skills to the development team of each product. "PSIRT is a dedicated team. If the teams are different, there is a possibility that information will be divided, so we think it is best to have a security officer on the product side. Ideally, each product should implement its own security and PSIRT itself should disappear," says Ms. Otsuka.
In order to increase the number of security-conscious engineers on the product development side, Gehirn also has a role to play.
Ms Otsuka: "The first thing we need to do is to increase the number of people who are interested in security. We need to invite developers in the company and say, 'We have these people doing security audits and finding these vulnerabilities. Isn't that interesting?'" In 2018, Cybozu and Gehirn jointly held a technical exchange meeting, hoping to raise awareness of the importance of this.
Based on the trusting relationship so far, and based on the contents of the report of the bug bounty program, this closed exchange meeting was held to exchange frank information about vulnerabilities. Gehirn's side gave a more pointed lightning talk than Otsuka expected, which was well received by the participants. Some people even said that Gehirn's image changed from an "audit company" to a "hacker company" after this event.
"I think this is another activity that will increase the number of security-conscious members," says Otsuka. He hopes to continue to share information in a passionate and in-depth way through this kind of forum, and to work with Gehirn to improve the security of its products and, more broadly, to create a secure society.
This article is based on an interview conducted in February of 2019.