Commit Graph

20 Commits

Author SHA1 Message Date
Aodhnait Étaín 569391612a
Add comment parsing to the lexer
Now lexer can properly parse and skip comments, as well as any
other combination of whitespace and comments.
2021-05-28 12:59:55 +00:00
Aodhnait Étaín 48f47671cd
Add minus operator
Adds "minus" operator, both as a unary negation and a binary
subtraction. Refactors Parser::bump to actually return the next
token, which is slightly more useful for us. Also changes how
Token::len is implemented, and now we return 1 for all tokens
that are not listed explicitly.
2021-05-27 17:40:36 +00:00
Aodhnait Étaín 56fffd6911
Add unary expressions
Adds parsing of unary plus to the parser.
2021-05-27 15:55:35 +00:00
Aodhnait Étaín 155325e78c
Add multiplication operator
Can now parse a multiplication operator with the correct precedence.
Parentheses are no longer included in the AST output.

Binary operator parser has been simplified to avoid redundancy, and to
avoid having to manually provide every token that can be a binary
operator.
2021-05-27 12:26:26 +00:00
Aodhnait Étaín 6e7b4d8319 Add parenthesized expression parsing
Another rewrite, this time to account for easier architecture. It also
adds parsing for simple binary expressions and parenthesized expressions
which was harder to do in the previous version.
2021-05-26 20:19:14 +01:00
Aodhnait Étaín 4e339b1f6e Rewrite to avoid using unsafe
For some reason in the previous version we had to use unsafe operations
on string slices because borrow checker would always complain. Somehow
after rewriting we don't need it anymore.
2021-05-25 18:59:12 +01:00
Aodhnait Étaín 5e2bdfa091 Remove parse_next from TokenStream public interface
Same reason as in c776f7.
2021-05-24 20:00:05 +01:00
Aodhnait Étaín d90f6763d5 Add additional documentation
Adds documentation for TokenStream's chars and skip_whitespace, for
OffsetStr, in particular describing why do we need that structure, and
can't just use Rust's builtin &str, for Offset::from, explaining the
unsafety behind calling this function, and describing how unsafe are
implementations of Display and Debug for OffsetStr.
2021-05-23 10:13:52 +01:00
Aodhnait Étaín c776f79f82 Make skip_whitespace private again
It was never supposed to be part of a public interface of TokenStream,
but it got added during debugging lifetimes issue. This commit finally
reverts it back to what it was supposed to be.
2021-05-23 10:11:33 +01:00
Aodhnait Étaín ab98d50a53 Rename GLOBAL_COUNTER to GRAPHVIZ_NODE_COUNTER
As GLOBAL_COUNTER could be confusing regarding what actually does it
count.
2021-05-23 10:08:53 +01:00
Aodhnait Étaín 66dc814b4b Remove unused code
It was only needed to test if the issue in 953938 was related to the
parser or the printer. As it's fixed now, we don't need it anymore.
2021-05-23 08:10:06 +01:00
Aodhnait Étaín 262df19aa3 Add multiplication to the expression parser
Now it also handles expression such as 11 * 13, 11 + 13 * 17, and even
11 + 13 * 17 + 19. We also introduced new function, is_binary_operator,
which allows for simpler checking of operators in the expression parser.

We fixed the issue that parser didn't parse its input exhaustively, i.e.
when given '+17 + 23 + +21' it would return only +17 + 23, which is far
from what we would expect.
2021-05-23 08:06:53 +01:00
Aodhnait Étaín a29dfc413c Add token caching to TokenStream
Now when calling peek, TokenStream will remember the token it parses and
store it in a field. Then, when calling next, it looks at that field and
if it's not empty, it returns it instead. This allows us to avoid having
to parse same token in the input two times, which ultimately results in
better performance.
2021-05-23 08:06:53 +01:00
Aodhnait Étaín 3062ac9f45 Add unary operator parsing
Now we can also parse unary +, i.e.
  +17,
and also expressions that contain it, i.e.
  +17 + 23.

We also now have custom Debug implementation for Expression, which
prints them in more useful, s-expression-like syntax, i.e.
  +17 + 23 => (+ +17 23).

We also change implementation of `TokenStream`s `next` and `parse_next`
methods to allow to easily implement `peek` method for looking at the
(possible) next token without advancing the stream.
2021-05-23 08:06:53 +01:00
Aodhnait Étaín 9539389e4f Fix segmentation fault
This segfault occurs in the future commits, but because it's more
important, I'm commiting it first.
2021-05-23 08:06:53 +01:00
Aodhnait Étaín fa2dcddd47 Rename CallWithContinuation trait to Then
New name is shorter and better reflects the intent of the trait.
2021-05-23 08:06:53 +01:00
Aodhnait Étaín 3d26398ac7 Add pretty-printing of the parsed expression
Adds `--unpretty` unstable flag, which currently accepts two arguments,
`dot` and `graphdotviz`, which are synonymous, and which prints the
expression we parsed in a graphdotviz-compatible format, i.e. in a form
of a directed graph.
2021-05-22 22:07:49 +01:00
Aodhnait Étaín ba409b9be0 Add simple expression parsing
Currently can only parse addition in a right-associative way, i.e.
  1 + 2 + 3 => 1 + (2 + 3).

The reason why are we using raw pointers in a form of OffsetStr struct
is because I don't think there is a way to prove to Rust compiler that
tokens the parse_next function returns are always valid references to
the TokenStream source string. Therefore we simply have to bypass the
borrow checker and handle this ourselves.
2021-05-22 21:45:42 +01:00
Aodhnait Étaín 4841f7b657 Add parsing of command-line arguments
Currently only supports `--help` argument and settings input file
implicitly from the positional arguments.
2021-05-22 21:38:34 +01:00
Aodhnait Étaín 37f58e24ac first commit, nyan~ 2021-05-22 21:34:55 +01:00