The 3rd 83 line code challenge in 2021 hosted by Alibaba cloud has ended. More than 20000 people watched, nearly 4000 people participated, and 85 teams came in groups. The competition adopts game breakthrough playing method, integrating meta universe science fiction and script killing elements, so that a group of developers have a lot of fun.
The last question of this competition tests the debugging ability of the contestants. It is best to have a certain understanding of spring weblux (at least spring mvc) and spring security based on springboot, which can save a lot of time to find information during the competition.
Next, from a perspective that is not so familiar with the above architecture, explain the debug idea step by step. It is still very helpful to understand spring in the future.
First step
Bug1
ReactiveWebSocketHandlerTest unit test debugging, first execute the unit test and view the execution results.
The call stack is analyzed from top to bottom. It is found that there is an EOF error, that is, the stream still tries to read after reading, and finds the source code context starting with com.aliyun.code83 in the error stack.
private static CheckedFunction<DataInputStream, String> charsetNameDecoder = (DataInputStream input) -> {
byte[] charsetNameBytes = input.readNBytes(input.readByte()); //Line 100 of the source code
if (charsetName.get() == null) {
charsetName.set(new String(charsetNameBytes, ISO_8859_1));
}
return charsetName.get();
};You can see that the error is in input.readByte() on line 100, but there are no other input operations above, which means that the stream has been read out when it is passed in. Continue to look up the error stack and find the context in line 89 of Utils.
public static String decodeMessage(byte[] rawMessage) {
ByteArrayInputStream in = new ByteArrayInputStream(rawMessage);
DataInputStream dis = new DataInputStream(in);
try {
return new String(dis.readAllBytes(), charsetNameDecoder.apply(dis)); //Source code line 89
} catch (IOException e) {
e.printStackTrace();
return String.format("%s<_>-<_>.", e.getClass().getSimpleName()); // Do not move this line, which will affect the score
}
}
From line 89, we can see that charsetNameDecoder.apply(dis) reports an error because the first parameter dis.readAllBytes() of new String has finished reading the data. Here we need to roughly analyze the functions of this part of the code.
new String if two parameters are passed in, the first parameter bytes [] is the byte array of the corresponding content, and the second parameter is the character set. It seems that both the string content and the character set come from the input input stream. From the logic of charsetNameDecoder, first read a length N through input.readByte, parse the character set in the part of reading the length N through readNBytes, and then read the rest to build the string according to the character set.
If you have read all the code completely, you can also see the package format from the javadoc of ReactiveWebSocketHandler, which also reflects the importance of javadoc. You may not have to read the code before doing the question, but try to read all the javadoc.
/**
* Binary package format
* byte Character set length; n1
* byte[n1] Character set data; n1 = character set length
* byte[n2] Valid data; n2 = total package length - n1 - 1
*/
@Component("ReactiveWebSocketHandler")
public class ReactiveWebSocketHandler implements WebSocketHandler {
...The problem here is obvious in line 89. readAllBytes has read all the data in advance, so the code is adjusted as follows
public static String decodeMessage(byte[] rawMessage) {
ByteArrayInputStream in = new ByteArrayInputStream(rawMessage);
DataInputStream dis = new DataInputStream(in);
try {
//First read the character set from the first part of the stream, and then read the rest through readAllBytes
final String charset = charsetNameDecoder.apply(dis);
return new String(dis.readAllBytes(), charset);
} catch (IOException e) {
e.printStackTrace();
return String.format("%s<_>-<_>.", e.getClass().getSimpleName()); // Do not move this line, which will affect the score
}
}Rerun the unit test and find that there are no errors in the ReactiveWebSocketHandlerTest (at least meet the judgment expectations of the unit test, and whether there are business errors is not necessarily true).
The second unit test, round4application tests, looks empty and succeeds directly to the next one.
Bug2
Perform the third unit test, UtilsTest.
The execution failure appears to be a string processing logic, and the processing result is not quite right.
Here's a test case:
Triplet.with(
"Extract normal text",
"Welcome to <pre>DevStudio</pre>",
"Welcome to DevStudio"
),
Triplet.with(
"extract CJK text",
"have<i>object</i>Are you ready? Don't panic, Give you one! Please add nail group to receive: <quote>35991139</quote>",
"Do you have an object? Don't panic, Give you one! Please add nail group to receive: 35991139"
),
Triplet.with(
"extract Tag text",
"<p>Cosy Have you ever used it, And search https://developer.aliyun.com/tool/cosy</p>",
"Cosy Have you ever used it, And search https://developer.aliyun.com/tool/cosy"
),
Triplet.with(
"Extract nesting tag text",
"<blockquote><p>401?!! Don't panic, don't worry, App Observer Help you~ https://help.aliyun.com/document_detail/326231.html learn about < / P > < / blockquote > ",
"401?!! Don't panic, don't worry, App Observer Help you~ https://help.aliyun.com/document_detail/326231.html learn about it“
),
Triplet.with(
"Halloween surprise small theater",
"<happy>All the green fried chicken were scattered, Early recovery of cervical and lumbar spine! </happy>Thief sincerity",
"All the green fried chicken were scattered, Early recovery of cervical and lumbar spine! Thief sincerity"
)
As can be seen from each use case, it seems that the expectation for processing logic is to remove all elements in brackets, just like eliminating html node definitions and retaining only text content. Here's a look at the method actually tested:
private static final Pattern REGULAR_HTML_TAG = Pattern.compile("<(?<tag>.*)>");
public static String stripHtmlTag(String html) {
if (ObjectUtils.isEmpty(html)) {
return null;
}
StringBuilder builder = new StringBuilder();
final Matcher matcher = REGULAR_HTML_TAG.matcher(html);
while (matcher.find()) {
matcher.appendReplacement(builder, Strings.EMPTY);
if (log.isDebugEnabled()) {
log.debug("remove tag {}", matcher.group("tag"));
}
}
return builder.toString();
}Indeed, the overall logic seems to match a pair of < >, and replace it with empty logic. First, analyze the regular expression defined at the top, which is OK at first glance, and match any characters with < >. *? It is used to name the matching group. It has no direct effect on matching. It is treated as the key of the group when replacing. For this, you can check the regular related documents. However, the UT execution is obviously wrong. Let's use this regular matching for a use case string
It can be seen that the regular matching goes from the first < directly to the last >, so the execution result is the whole sentence replacement with a "you" word left. This involves the regular greedy matching problem. It defaults to greedy and matches more content as much as possible. The way to cancel greedy is to add a question mark after the matching rule? become
private static final Pattern REGULAR_HTML_TAG = Pattern.compile("<(?<tag>.*?)>");bug3
Execute UT again after changing
It looks much better! There is only one mistake. Now let's analyze why it is wrong.
public static String stripHtmlTag(String html) {
if (ObjectUtils.isEmpty(html)) {
return null;
}
StringBuilder builder = new StringBuilder();
final Matcher matcher = REGULAR_HTML_TAG.matcher(html);
while (matcher.find()) {
matcher.appendReplacement(builder, Strings.EMPTY);
if (log.isDebugEnabled()) {
log.debug("remove tag {}", matcher.group("tag"));
}
}
return builder.toString();
}From the loop point of view, tag matching is performed for html. If it is found, a new string is written to the builder, and the content of the new string is the text up to the matching part, and < (. *?) > is replaced with empty. For the use case of "green oil chicken all retreats, cervical and lumbar vertebrae recover as soon as possible! Thief is sincere".
• the first matching content is < happy >, and the replacement text written to the builder is empty;
• the second matching part is that all the green oil chickens are scattered and the cervical and lumbar vertebrae recover as soon as possible</ Happy >, the content added to the builder is "all Bi Youji retreat and disperse, and the cervical and lumbar vertebrae will recover as soon as possible!"
• the third time while comes, because there is no new <. * > content, just end the loop and return! So the problem is here. We need to make up the rest of the "thief sincerity".
So the code is adjusted as follows:
public static String stripHtmlTag(String html) {
if (ObjectUtils.isEmpty(html)) {
return null;
}
StringBuilder builder = new StringBuilder();
final Matcher matcher = REGULAR_HTML_TAG.matcher(html);
while (matcher.find()) {
matcher.appendReplacement(builder, Strings.EMPTY);
if (log.isDebugEnabled()) {
log.debug("remove tag {}", matcher.group("tag"));
}
}
matcher.appendTail(builder);
return builder.toString();
}Add appendTrail to make up the rest. Run the unit test, everything is OK!
In fact, there is another simple way to write it:
public static String stripHtmlTag(String html) {
if (ObjectUtils.isEmpty(html)) {
return null;
}
return html.replaceAll("<.*?>", "");
}However, since you change the bug, try to keep the original logic.
Step 2
Now let's start business debugging and run it according to the README prompt/ round4. I hung up at the beginning. I need to start the service when I see the reminder.
Find the main method with @ SpringBootApplication annotation, which is the standard startup entry of spring boot program. You can start, run/debug. If you want breakpoint debugging, use debug.
Bug4
Execute. / round4 again
There seems to be nothing wrong with step 1. There seems to be an error in step 2. It seems that it is expected to dynamically add a user reporter, but it fails. The error message is the lack of CSRF request header. If spring security (whether in spring MVC or spring Webflux) is used, an image may be left in the security configuration, that is, the configuration of CSRF. Here, we can see that the round4 client seems to request a token without CSRF, so we can only change the service.
Note: csrf Baidu can understand its role, purpose and basic mechanism. spring security has a native implementation. As long as it is processed through configuration, the csrf function is enabled by default.
Find the security configuration class WebSecurityConfig, and adjust the configuration as follows:
@Bean
public SecurityWebFilterChain securityFilterChain(ServerHttpSecurity http) {
return http
.headers().disable()
.authorizeExchange()
.pathMatchers("/endpoints").hasAnyRole("USER")
.pathMatchers("/users").hasAnyRole("admin")
.pathMatchers("/ws/test").hasAnyRole("TEST") // Do not change this line, otherwise the score will be affected
.pathMatchers("/ws/**").hasAnyRole("admin")
.anyExchange().authenticated()
.and()
.httpBasic()
.and()
.formLogin().disable()
//Join this line
.csrf().disable()
.build();
}bug5
Restart the service and execute. / round4
The second step hung up again, but the error content changed to 401. The description is that the identity certificate is wrong. The error log lovingly typed the error certificate content in basic auth mode, followed by a string of characters.
A look at the garbled characters at the end of = will easily associate with base64. Just find a base64 decryption tool and put this string of words in it.
The solution shows a typical account: password format, that is, the attempt to use admin / admin123 as the account password failed. Go back to the WebSecurityConfig class and check the configuration
@Bean
public MapReactiveUserDetailsService userDetailsService() {
UserDetails user = User.builder()
.username("user")
.password("{noop}user")
.roles("USER")
.build();
UserDetails admin = User.builder()
.username("admin")
.password("{noop}admin")
.roles("ADMIN")
.build();
return new MapReactiveUserDetailsService(user, admin);
}When you see the configuration of admin, the password seems to be admin. As for what {noop} means, if you have the energy to debug, you can follow it. UserDetailService uses a PasswordEncoder interface for password management, because the password is entered in clear text, but for security reasons, the password can only be stored after being confused in the database, Otherwise, the consequences of database data leakage will be disastrous. PasswordEncoder has many implementation classes. UserDetailService uses a class called DelegatingPasswordEncoder by default. It will give plaintext to different passwordencoders for ciphertext matching according to the situation, and this "situation" is the content of the preceding braces. The following are various obfuscation algorithms for DelegatePasswordEncoder registration.
public final class PasswordEncoderFactories {
private PasswordEncoderFactories() {
}
public static PasswordEncoder createDelegatingPasswordEncoder() {
String encodingId = "bcrypt";
Map<String, PasswordEncoder> encoders = new HashMap();
encoders.put(encodingId, new BCryptPasswordEncoder());
encoders.put("ldap", new LdapShaPasswordEncoder());
encoders.put("MD4", new Md4PasswordEncoder());
encoders.put("MD5", new MessageDigestPasswordEncoder("MD5"));
encoders.put("noop", NoOpPasswordEncoder.getInstance());
encoders.put("pbkdf2", new Pbkdf2PasswordEncoder());
encoders.put("scrypt", new SCryptPasswordEncoder());
encoders.put("SHA-1", new MessageDigestPasswordEncoder("SHA-1"));
encoders.put("SHA-256", new MessageDigestPasswordEncoder("SHA-256"));
encoders.put("sha256", new StandardPasswordEncoder());
encoders.put("argon2", new Argon2PasswordEncoder());
return new DelegatingPasswordEncoder(encodingId, encoders);
}
}It can be seen that noop corresponds to an instance called NoOpPasswordEncoder, that is, no operation. The plaintext does not do anything and is directly stored or compared in plaintext, so it is also convenient for us to modify.
In short, {noop} can see what's going on, but don't touch what you can't see. Just touch what you understand. Admin still knows it. Change it to {noop}admin123.
Bug6
Restart the service and execute. / round4
Step 2 another error continues to hang up
This time, I saw the wrong permissions, but I can't see what permissions are needed. At this point, we go back to the Web services console for clues
In the service log, after trying to call POST /users, a 403 error is reported, which should be the same as the error of round4, that is, there may be a problem with the permissions of the / users interface. Go back to the code to check
public class WebSecurityConfig {
@Bean
public SecurityWebFilterChain securityFilterChain(ServerHttpSecurity http) {
return http
.headers().disable()
.authorizeExchange()
.pathMatchers("/endpoints").hasAnyRole("USER")
.pathMatchers("/users").hasAnyRole("admin")
.pathMatchers("/ws/test").hasAnyRole("TEST") // Do not change this line, otherwise the score will be affected
.pathMatchers("/ws/**").hasAnyRole("admin")
.anyExchange().authenticated()
.and()
.httpBasic()
.and()
.formLogin().disable()
.csrf().disable()
.build();
}
@Bean
public MapReactiveUserDetailsService userDetailsService() {
UserDetails user = User.builder()
.username("user")
.password("{noop}user")
.roles("USER")
.build();
UserDetails admin = User.builder()
.username("admin")
.password("{noop}admin123")
.roles("ADMIN")
.build();
return new MapReactiveUserDetailsService(user, admin);
}
}From the configuration, the admin account has a role called "admin", and the configuration of the above / users interface requires the permission "admin". Unlike the database, which is insensitive to case and has strict identity permissions, you can't make a difference in a letter. Change all the admin above to uppercase admin.
Bug7
Restart the service and try again
Congratulations! Step 2 ran through. No matter what it did, it ran through! Step 3 is also a permission problem, but we don't know what the problem is. If the account password is wrong, it's 401, and the reporter is dynamically added in step 2. If the password doesn't match, there's no way to adjust it.
Judging from the debugging experience of step 2 just now, the reason for 403 Forbidden is probably the permission, but we don't know what the permission of the reporter is. At this time, we need to make a breakpoint for / users to see what step 2 put in.
Find the Round4Controller, make a breakpoint in the addUser method, and execute. / round4
You can see that the calling interface of step 2 is a parameter, username is reporter, password is reporter, and there is a special field called authorities, with the content of ROLE_REPORTER, which looks suspicious, seems to grant permissions to this user.
The interfaces called by Step3 are / ws/DevStudio, / ws/Cosy and WebSecurityConfig. The matching rules are
.pathMatchers("/ws/**").hasAnyRole("ADMIN")According to the current server configuration, only the ADMIN role can be accessed, so you need to open a back door for it. See that hasAnyRole is an array type parameter and add a value "ROLE_REPORTER"
.pathMatchers("/ws/**").hasAnyRole("ADMIN","ROLE_REPORTER")Try again, this time it should be... Hanging up again! The mistake is still the same?
That means the role just added is wrong?
At this time, observe the configuration. Other roles are called ADMIN, USER and TEST, which is preceded by a Role_ It's strange, and in POST /users, this value is placed in an array called authorities. The name is very broad and does not refer to Role specifically. Will the previous Role be the same as {noop}ADMIN123 just now_ Is it a hidden rule? If it's a license_ Or something like that? So, this Role may be called REPORTER, readjusting the configuration
.pathMatchers("/ws/**").hasAnyRole("ADMIN", "REPORTER")Try again! Congratulations, there has been a change! And the prompt can be scored. Call. / round4 --submit. After ignoring the three soul problems, there are finally scores, but there are obviously a lot of problems. It seems that it just went through, but the result is not satisfactory.
Bug8
Re execute. / round4 observe the output, and you can see that there are full of random codes, and there is almost no normal Chinese. Combined with the previous Unit Test debugging, you can guess that this may be related to character set processing, that is, there should still be bugs in Utils.
At this time, we can probably see the purpose of this program from the output. It seems that it is to accept the request of round4 and output some text. These business interface entries are / ws / * *.
Start to find out how / ws / * * is mapped to these methods. First, find the suspicious methods in WebConfig
@Autowired
@Qualifier("ReactiveWebSocketHandler")
private WebSocketHandler webSocketHandler;
@Bean
public Map<String, WebSocketHandler> webSocketUrlMap() {
return Utils.randomWords(3)
.stream()
.map(w -> "/ws/" + w)
.collect(Collectors.toMap(Function.identity(), w -> webSocketHandler));
}It seems that a Bean of Map type is registered. The key is the address starting with / ws, and the value is WebSocketHandler. In the definition of WebSocketHandler, it is declared that its name is ReactiveWebSocketHandler in the spring container. Then, I searched the text of ReactiveWebSocketHandler and found another suspicious class.
Component("ReactiveWebSocketHandler")
public class ReactiveWebSocketHandler implements WebSocketHandler {
@Override
public Mono<Void> handle(WebSocketSession session) {
return session.send(
session.receive()
.map(WebSocketMessage::getPayload)
.map(getBufferConverter())
.map(Utils::decodeMessage)
.map(Utils::stripHtmlTag)
.log()
.map(session::textMessage));
}
private Function<DataBuffer, byte[]> getBufferConverter() {
final byte[] buffer = new byte[1024];
return (DataBuffer dataBuffer) -> {
int length = dataBuffer.readableByteCount();
dataBuffer.read(buffer, 0, length);
return buffer;
};
}
}It seems very relevant. The handle method seems to deal with text. As for what Mono is, baidu is related to WebFlux. Maybe it doesn't understand it very well, but looking at a series of map methods, if you are familiar with the characteristics of java8, it is very similar to those behind Collections.stream() or in the Optional class. The use of map is a series of mapping logic. Guess their functions from the method name:
- WebSocketMessage::getPayload obtains the request body;
- getBufferConverter() is converted into a buffer;
- Utils::decodeMessage decoding;
- Utils::stripHtmlTag remove tag;
- log() print log;
- session::textMessage outputs text to the session;
Among them, 1 and 6 are spring methods, and there is little possibility of bug s. Generally, there is nothing wrong with log. The problem may be above 2, 3 and 4.
First of all, there is a large amount of random code. I feel that it is inseparable from decodeMessage. Analyze the source code:
public static String decodeMessage(byte[] rawMessage) {
ByteArrayInputStream in = new ByteArrayInputStream(rawMessage);
DataInputStream dis = new DataInputStream(in);
try {
final String charset = charsetNameDecoder.apply(dis);
return new String(dis.readAllBytes(), charset);
} catch (IOException e) {
e.printStackTrace();
return String.format("%s<_>-<_>.", e.getClass().getSimpleName()); // Do not move this line, which will affect the score
}
}
private static final ThreadLocal<String> charsetName = new ThreadLocal<>();
private static final CheckedFunction<DataInputStream, String> charsetNameDecoder = (DataInputStream input) -> {
byte[] charsetNameBytes = input.readNBytes(input.readByte());
if (charsetName.get() == null) {
charsetName.set(new String(charsetNameBytes, ISO_8859_1));
}
return charsetName.get();
};Unit Test has been done for these two classes, which shows that the hard damage is not large, but the number of lines of code is small. Analyze it slowly.
The logic of decodeMessage is relatively clear. It seems that there is no big problem. Continue to look at charsetNameDecoder.
charsetNameDecoder uses a ThreadLocal to store information. It may not be clear whether WebSocket uses a new thread to process every request. It is not clear whether ThreadLocal will be polluted due to thread pool reuse, but at least I have worked hard to resolve charset. I don't need to parse it just because there is data in ThreadLocal.
From the meaning of this code, it seems that if charset can be parsed, it is best. If it cannot be parsed, use the previously parsed one, so adjust the code according to this idea:
private static final CheckedFunction<DataInputStream, String> charsetNameDecoder = (DataInputStream input) -> {
byte[] charsetNameBytes = input.readNBytes(input.readByte());
if (charsetNameBytes != null && charsetNameBytes.length > 0) {
charsetName.set(new String(charsetNameBytes, ISO_8859_1));
}
return charsetName.get();
};Try again!
Bug9
Congratulations. It can be seen that the garbled code in the result is significantly reduced, and there is normal Chinese, indicating that the modification of the character set has an effect, and from the output, it seems to be trying to output some java code.
Carefully observe the law of garbled code. It seems that it appears at the end of each line. The front part of garbled code looks ok. Can it be the reason why the flow is too long?
Go to the previous step ReactiveWebSocketHandler::getBufferConverter
private Function<DataBuffer, byte[]> getBufferConverter() {
final byte[] buffer = new byte[1024];
return (DataBuffer dataBuffer) -> {
int length = dataBuffer.readableByteCount();
dataBuffer.read(buffer, 0, length);
return buffer;
};
}It seems to build an array with a length of 1024, and then fill in the content of length... And so on? Why do you fix 1024 when you know the length? Is October 24th lucky or something?
Change and run!
private Function<DataBuffer, byte[]> getBufferConverter() {
return (DataBuffer dataBuffer) -> {
int length = dataBuffer.readableByteCount();
final byte[] buffer = new byte[length];
dataBuffer.read(buffer, 0, length);
return buffer;
};
}Bug10
The random code has disappeared! It seems all OK! At this time, when you glance at the web service console, why do you output a bunch of errors?
Classic error NPE. Looking at onNext in this row, I feel that it has something to do with the map in that row. Can't the data passed down be null? Check each step and find the suspicious code in Utils::stripHtmlTag.
if (ObjectUtils.isEmpty(html)) {
return null;
}
StringBuilder builder = new StringBuilder();
final Matcher matcher = REGULAR_HTML_TAG.matcher(html);
while (matcher.find()) {
matcher.appendReplacement(builder, Strings.EMPTY);
if (log.isDebugEnabled()) {
log.debug("remove tag {}", matcher.group("tag"));
}
}
matcher.appendTail(builder);
return builder.toString();
}
If the incoming html is empty, it returns a null. Why? Change the null to "" and try again.
Finally, there are no random codes and errors. Submit the score! 90 points!
Keep looking for bug s! After looking for it for 10 minutes, I didn't see what was wrong. Then I had no choice but to look at README again and found that the last 10 points were the scores of those three questions
Don't look at the documents and kill people
As for the last three questions, I tried them out slowly. Please see the interpretation of the organizer for the correct answer
last
In actual work, it is taboo to debug without reading the code to find out the function, because round4 usually doesn't give us a score, it's easy to get rid of a bug and bring a group of new ones.
Therefore, the above strategy is based on the competition environment and the pursuit of speed when there is a clear scoring system. It is not recommended to use it in daily work. Of course, the idea of looking for errors in debug is common.
If you have enough time to understand the business background (the competition will not give detailed prd...) and technical ideas, most of these bugs can be found and eliminated directly with the naked eye. Of course, there will not be so many low-level bugs at the bottom in your daily work. If you face such bugs every day, architects can sacrifice flags
At present, all levels of the competition are open for experience. Domain name and address: https://code83.ide.aliyun.com/ , welcome.
Recommended reading
1,Play script with code? Official analysis of the plot of the 3rd 83 line code competition
2,No algorithm, no Java, this algorithm problem is difficult?
Welcome to use Cloud effect, cloud native era, new DevOps platform , greatly improve R & D efficiency through new cloud native technologies and new R & D models. At present, the cloud effect public cloud basic version is not limited to 0 yuan.