REBOL3 tracker
  0.9.12 beta
Ticket #0000537 User: anonymous

Project:



Short URL: http://issue.cc/r3/537
rss
TypeWish Statusdeferred Date16-Jan-2009 21:40
Versionalpha 31 CategorySyntax Submitted byBrianH
PlatformAll Severitytweak Priorityhigh

Summary TRANSCODE/error tweak with these characters: #$%,<>\
Description If TRANSCODE could treat a comma as a delimiter, like it treats semi-colons, it would greatly enhance its usefulness. Wait to throw the error until the next time. This could also extend to all of these characters: "#$%,<>\"
Example code
Current behavior:
>> transcode/error to-binary "a,"
== [make error! [
        code: 200
        type: 'Syntax
        id: 'invalid
        arg1: "word"
        arg2: "a,"
        arg3: none
        near: "(line 1) a,"
        where: [transcode]
    ] #{}]
>> transcode to-binary "a,0"
** Syntax error: invalid "word" -- "a,0"
>> to-block "a,0"
** Syntax error: invalid "word" -- "a,0"

Desired behavior:
>> transcode/error to-binary "a,"
== [a make error! [
        code: 200
        type: 'Syntax
        id: 'invalid
        arg1: "word"
        arg2: ","
        arg3: none
        near: "(line 1) a,"
        where: [transcode]
    ] #{}]
>> equal? reduce ['a to-binary ","] transcode/next to-binary "a,"
== true
>> transcode/next to-binary "a,"
== [a #{2C}]
>> transcode to-binary "a,0"
== [a 0.0]
>> to-block "a,0"
== [a 0.0]

And the effect on LOAD:
>> load "a,0"
== [a 0.0]

Assigned ton/a Fixed in- Last Update30-Mar-2015 01:13


Comments
(0000361)
Carl
23-Jan-2009 07:49

Is this request just for words? That may be possible, but in general comma cannot be a super-delimiter because it is valid in a range of datatypes - strings, files, issues, urls, and even decimal numbers.
(0000368)
BrianH
23-Jan-2009 17:39

Just for words would be sufficient, yes.
(0001451)
BrianH
31-Jul-2009 23:49

Updated the example code and summary to reflect the alpha 39 changes to TRANSCODE.
(0001979)
Carl
6-Feb-2010 20:14

http://www.rebol.net/r3blogs/0306.html for discussion.
(0002211)
BrianH
19-Apr-2010 19:22

It has been suggested that there be a place in system/catalog where standard charsets that would be used by many parse rules would be stored. If a word-char charset was one of those, including all of the chars that are recognized as being part of a word! (not including the special cases), then the request for the comma tweak to TRANSCODE would be unnecessary, and could be withdrawn.

Heck, even documenting the R3 syntax with an optional module filled with charsets and parse rules - kept current with the precise syntax - would be enough. That would allow us to write a whole variety of R3 syntax analysis tools, recovery parsers and such.
(0003114)
BrianH
17-Feb-2011 10:11

The beginnings of a set of documented word charsets and rules have been posted in a comment to #1302. If we decide to make this tweak with TRANSCODE, it would be best for it to apply to all of the characters in this string: "#$%,<>\"
(0003118)
BrianH
17-Feb-2011 22:29

The upside of making this change would be that we wouldn't have to write as many spaces after words; the next value would be recognized immediately. The downside would be that we wouldn't have to write spaces after words, so those spaces wouldn't be written, which would lead to reduced readability. It would be tougher for the programmer to tell when the word ends and the next value begins. Tradeoff?
(0004618)
MarkI
30-Mar-2015 01:13

If non-word characters close a word token, no syntax extensions that begin with a word would ever be backward-compatible.

Date User Field Action Change
30-Mar-2015 01:13 MarkI Comment : 0004618 Added -
17-Feb-2011 22:30 BrianH Comment : 0003118 Modified -
17-Feb-2011 22:29 BrianH Comment : 0003118 Added -
17-Feb-2011 10:13 BrianH Summary Modified TRANSCODE/error tweak with commas => TRANSCODE/error tweak with these characters: #$%,<>\
17-Feb-2011 10:13 BrianH Description Modified -
17-Feb-2011 10:11 BrianH Comment : 0003114 Added -
19-Apr-2010 19:22 BrianH Comment : 0002211 Added -
6-Feb-2010 20:15 Carl Status Modified reviewed => deferred
6-Feb-2010 20:14 Carl Comment : 0001979 Added -
1-Aug-2009 03:37 BrianH Code Modified -
1-Aug-2009 00:26 BrianH Code Modified -
1-Aug-2009 00:24 BrianH Code Modified -
1-Aug-2009 00:20 BrianH Code Modified -
31-Jul-2009 23:50 BrianH Comment : 0001451 Modified -
31-Jul-2009 23:49 BrianH Comment : 0001451 Added -
31-Jul-2009 23:48 BrianH Summary Modified TRANSCODE tweak with commas => TRANSCODE/error tweak with commas
31-Jul-2009 23:48 BrianH Code Modified -
31-Jul-2009 23:14 BrianH Category Modified => Syntax
26-Feb-2009 03:35 BrianH Priority Modified normal => high
23-Jan-2009 17:39 BrianH Comment : 0000368 Added -
23-Jan-2009 07:49 carl Comment : 0000361 Added -
19-Jan-2009 22:11 BrianH Code Modified -
19-Jan-2009 22:11 BrianH Severity Modified minor => tweak
19-Jan-2009 22:11 BrianH Status Modified submitted => reviewed
16-Jan-2009 21:40 BrianH Ticket Added -