"D is a systems programming language with C-like syntax and static typing. It combines efficiency, control and modeling power with safety and programmer productivity"
-dlang.org
First, let me tell you how I stumbled upon D language: I was upset by how C# source code can be retained from it's executable (or DLL) no matter if you use obfuscators and such. I was thinking why is it that much hard to interact with native APIs and being unable to do some low level stuff in C# was not fun. There was few other things I had in mind wondering why can't there be that kind of language... I tried my best to like C++, I really did. No matter how much I tell myself it's not that bad, I gave up. It's unnecessarily complicated with no win. So I started to look for some C like language that can handle anything from low level to high level, have an optional GC, feature rich, and plays well with other languages. After weeks of searching, I said to myself "I bet there is a language called D" and I googled "d programming language". First result was the dlang.org. When I clicked on it, I was no longer thinking about creating my own language. It was there, it was D.
Module system
No painful header file sh*t. There is no confusion of declaration and definition, order of signatures don't have a side effect, and no need for maintaining two different signatures. Yet, D supports separate compilation in a much cleaner way than C/C++.
Modules are simple:
Doing something similar to above will require you to name your file "algorithm.d" and put it in a directory called "drone". This way you're forced to do the right thing. Doing so makes other people to figure out your code easier and it makes the project more manageable in the long run.
Here is how one can use modules:
Compile times
The official reference compiler DMD is incredibly fast. One reason is not having any preprocessor (hence the module system), the other one is Walter Bright's outstanding optimization skills. DMD doesn't usually produce highly optimized/fast code, it produce it fast. Thus, compile-run-test cycle becomes satisfying. Many C++ guys expect a compiler error in their first D code compilation but get surprised that it's successfully compiled at such speed. There are also LDC (LLVM based D compiler) and GDC (GCC based D compiler) which compiles slower but produces strongly optimized binaries.
Slices
A slice in D seems like a dynamic array in almost all aspects of the concept -- when passed without adornments, the data referred to is passed by reference, and it supports all the properties and functions one would expect a dynamic array type to support. But there is one very important difference. A slice does not own the array, it references the array. That is, the slice is not responsible for allocation or deallocation of its data. The responsible party for managing a dynamic array's memory is the D runtime.
So where is the true dynamic array type in D? It's hidden by the runtime, and in fact, has no formal type. Slices are good enough, and as it turns out, the runtime is smart enough about what you want to do with the data, that you almost never notice dynamic arrays are missing as a full-fledged type. In fact, most D coders consider the D slice to be the dynamic array type -- it's even listed as a dynamic array type in the spec!
With the combined protection of having the length of the data, and the garbage collector to manage the memory backing the data, slices are an extremely powerful, dynamic concept that is safe from most memory corruption issues. With D slices, one can write high-performance code with elegant and concise syntax that is awkward or inefficient in almost any other language.
Let's see some D slices in action:
reference: D Array Article
Here is an advanced LF2 data parser (tokenizer) I wrote a while ago, it runs blazingly fast because no string copying takes place:
Compile time function evaluation
This will describe the basics of a very powerful feature of the D programming language: the Compile-time Function Execution (CTFE), which allows complicated functions to be fully evaluated at compile-time, irrespective of the optimization levels.
C optimizations
If you are an average C programmer, you know that simple code can be trusted to be evaluated at compile-time thanks to optimizers. For instance, if you write something like:
You trust your compiler to evaluate the square at compile-time, when optimizations are on. When things get more hairy:
C programmers get immediately less confident about what will happen at run-time. For instance, would you say that your compiler is able to expand the above code at compile-time or not? Actually, the answer is “yes” in this particular case (unless you are using a very old compiler), but the point is still valid: this is not C code that one would write if he wants to be sure that the whole calculation be folded at compile-time.
There is also another issue: since the language does not mandate that the value is folded (and in fact, it is not folded when optimizations are disabled), you cannot create a constant out of it, such as by assigning it to a const variable.
When things get hairy
Now, let’s try with a (very naive and simple) solution of problem #1 of Project Euler:
This program simply calculates the sum of all divisors of 3 or 5 below 1000. But if you look at the generated code with GCC under -O3, you will see that the actual results are not computed at compile-time, but rather calculated at runtime. I believe any average C programmer would agree that we should not expect this code to be folded at compile time.
Now, meet the equivalent D code:
Deja-vu? Yes, it is exactly the same, barring the initial include statement that is not required (actually, there is no preprocessor in D and modules refer to each other with the import statement, but printf is a builtin). Of course, the above example was hand-crafted to make it both valid C and D code, but being D an evolution of C, the basic syntax is the same.
Meet CTFE
And now the hattrick: in D, we can request the compiler to evaluate euler1 at compile-time by simply using the static keyword at invocation time:
Great, isn’t it? Now the result of the above function call are evaluated by the compiler, irrespective of the optimization levels. If the function cannot be evaluated at compile-time (usually because it has side-effects, like any kind of I/O), it will trigger a compile-time error.
We can verify that the above constants really do appear in the generated code by compiling with gdc -save-temps euler1.d and then inspecting euler1.s:
Notice how the compiler has calculated the values 23 and 233168 (respectively, results of euler1(10) and euler1(1000)) and put them in the data section of the executable.
If you are curious of what happens when the compiler cannot do the whole evaluation at compile-time, it is sufficient to stick a printf() call somewhere in the euler() function. Since printf() does some I/O, it breaks CFTE, and the compiler will happily tell you about that:
CTFE is simple yet very powerful. The fact that it is triggered at the call site (rather than being an attribute of the function, like the inline keyword) is a very smart design choice: it makes perfectly sense for the same function to be used at both run-time and compile-time, depending on the inputs.
For the C++ guys reading, C++0x has grown a constexpr keyword that, while looking superficially similar, it is a lot less powerful, since it can only be used on very simple functions (basically, one-liners). In fact, the keyword is meant to be used while declaring a function, and not at the call-site, so it has to apply only on small functions which can be proved to always yield a constant value.
origin: Compile Time Function Execution in D
Templates, Meta-programming, mixins, and CTFE combination
Meta-programming in D is extremely powerful, yet easy to use and understand at the same time.
Writing a generic code that works with any type is dead simple:
Imagine the possibilities:
As this post turned out to be too long, you can ask me to elaborate on things you couldn't understand.
Have a nice day...
-dlang.org
First, let me tell you how I stumbled upon D language: I was upset by how C# source code can be retained from it's executable (or DLL) no matter if you use obfuscators and such. I was thinking why is it that much hard to interact with native APIs and being unable to do some low level stuff in C# was not fun. There was few other things I had in mind wondering why can't there be that kind of language... I tried my best to like C++, I really did. No matter how much I tell myself it's not that bad, I gave up. It's unnecessarily complicated with no win. So I started to look for some C like language that can handle anything from low level to high level, have an optional GC, feature rich, and plays well with other languages. After weeks of searching, I said to myself "I bet there is a language called D" and I googled "d programming language". First result was the dlang.org. When I clicked on it, I was no longer thinking about creating my own language. It was there, it was D.
Module system
No painful header file sh*t. There is no confusion of declaration and definition, order of signatures don't have a side effect, and no need for maintaining two different signatures. Yet, D supports separate compilation in a much cleaner way than C/C++.
Modules are simple:
D-Code:
module drone.algorithm; int[] quickSort(int[] data) { //Implementation... } |
Doing something similar to above will require you to name your file "algorithm.d" and put it in a directory called "drone". This way you're forced to do the right thing. Doing so makes other people to figure out your code easier and it makes the project more manageable in the long run.
Here is how one can use modules:
D-Code:
module program; import drone.algorithm : quickSort; //import only the required function import stdio = std.stdio; //renamed import can help not to pollute global scope import std.algorithm; //just simply import void main(string[] args) { import std.conv : to; //scoped imports will only available in the function scope foreach(e; quickSort(args.map!(a => a.to!int))) stdio.writeln(e); } |
Compile times
The official reference compiler DMD is incredibly fast. One reason is not having any preprocessor (hence the module system), the other one is Walter Bright's outstanding optimization skills. DMD doesn't usually produce highly optimized/fast code, it produce it fast. Thus, compile-run-test cycle becomes satisfying. Many C++ guys expect a compiler error in their first D code compilation but get surprised that it's successfully compiled at such speed. There are also LDC (LLVM based D compiler) and GDC (GCC based D compiler) which compiles slower but produces strongly optimized binaries.
Slices
A slice in D seems like a dynamic array in almost all aspects of the concept -- when passed without adornments, the data referred to is passed by reference, and it supports all the properties and functions one would expect a dynamic array type to support. But there is one very important difference. A slice does not own the array, it references the array. That is, the slice is not responsible for allocation or deallocation of its data. The responsible party for managing a dynamic array's memory is the D runtime.
So where is the true dynamic array type in D? It's hidden by the runtime, and in fact, has no formal type. Slices are good enough, and as it turns out, the runtime is smart enough about what you want to do with the data, that you almost never notice dynamic arrays are missing as a full-fledged type. In fact, most D coders consider the D slice to be the dynamic array type -- it's even listed as a dynamic array type in the spec!
With the combined protection of having the length of the data, and the garbage collector to manage the memory backing the data, slices are an extremely powerful, dynamic concept that is safe from most memory corruption issues. With D slices, one can write high-performance code with elegant and concise syntax that is awkward or inefficient in almost any other language.
Let's see some D slices in action:
D-Code:
import std.stdio; void main() { int[] a; // a is a slice a = new int[5]; // allocate a dynamic array of integers that has at least 5 elements, and give me a slice to the first 5. Note that all data in D is default assigned, int's are defaulted to 0, so this array contains five 0's int[] b = a[0..2]; // This is a 'slicing' operation. b now refers to the first two elements of a. Note that D uses open interval for the upper limit, so a[2] is not included in b. int[] c = a[$-2..$]; // c refers to the last two elements of a ($ stands for length inside a slice or index operation). c[0] = 4; // this also assigns a[3] c[1] = 5; // this also assigns a[4] b[] = c[]; // assign the first two elements of a[] to the value from the last two elements (4, 5). writeln(a); // prints "[4, 5, 0, 4, 5]" int[5] d; // d is a fixed sized array, allocated on the stack b = d[0..2]; // slices can point at fixed sized arrays too! } |
reference: D Array Article
Here is an advanced LF2 data parser (tokenizer) I wrote a while ago, it runs blazingly fast because no string copying takes place:
D-Code:
import std.stdio : File; import std.traits : isSomeString, isArray; import std.range : isInfinite, isIterable, isInputRange; import std.string : startsWith; enum TokenState : ubyte { none, xml, token, comment } enum TokenType : ubyte { normal, xml, property, } struct Token(S) if(isSomeString!S) { S str; size_t line, col; TokenType type; bool commentic; string toString() { return format(`"%s"[line: %d col: %d] `, str, line, col); } } immutable string tokenHeads = ['<'], tokenEnds = ['>', ':'], tokenDelims = [' ', '\t'], lineEnds = ['\n', '\r']; enum char lineCommentChar = '#'; class ParserException : Exception { this(string msg, string file = __FILE__, size_t line = __LINE__, Throwable next = null) @(safe, nogc) pure nothrow { super(msg, file, line, next); } } /// This function tokenizes LF2 data and returns a slice-array of strings. Returned slices point to the given string. public Token!S[] parseData(S)(S data, bool includeComments = false) if(isSomeString!S) { debug(LogFile) { File parserLog = File("parser.log", "wb"); scope(exit) parserLog.close(); } Token!S[] slices = new Token!S[4]; slices.reserve(data.length / 5); // Pre-allocate an aprox memory we might need bool commentness = false; TokenState state = TokenState.none; size_t tokenStart = 0, tokenCol = 1, tokenLine = 1, line = 1, col = 1; Lforeach: foreach(i, ch; data) { Lswitch: final switch(state) { case TokenState.none: if(lineEnds.canFind(ch)) { commentness = false; break Lswitch; } else if(tokenDelims.canFind(ch)) { break Lswitch; } else if(tokenHeads.canFind(ch)) // < { state = TokenState.xml; tokenStart = i; tokenCol = col; tokenLine = line; } else if(tokenEnds.canFind(ch)) // > : { throw new ParserException(format("Unexpected token ending delimeter: '%c' in line: %d; at col: %d", ch, line, col)); } else if(ch == lineCommentChar) // # { commentness = true; if(!includeComments) state = TokenState.comment; } else { state = TokenState.token; tokenStart = i; tokenCol = col; tokenLine = line; } break Lswitch; case TokenState.xml: if(lineEnds.canFind(ch)) { throw new ParserException(format("Unexpected line ending in line %d; at col %d", line, col)); } else if(tokenDelims.canFind(ch)) { throw new ParserException(format("Unexpected token delimeter in line %d; at col %d", line, col)); } else if(tokenHeads.canFind(ch)) // < { throw new ParserException(format("Unexpected token beginning delimeter '%c' in line %d; at col %d", ch, line, col)); } else if(tokenEnds[0] == ch) // > { slices ~= Token!S(data[tokenStart .. i + 1], tokenLine, tokenCol, TokenType.xml, commentness); state = TokenState.none; } else if(tokenEnds[1] == ch) // : { throw new ParserException(format("Unexpected token ending delimeter '%c' in line %d; at col %d", ch, line, col)); } else if(ch == lineCommentChar) // # { commentness = true; if(!includeComments) throw new ParserException(format("Unexpected comment char '%c' in line %d; at col %d", ch, line, col)); } break Lswitch; case TokenState.token: if(lineEnds.canFind(ch)) { slices ~= Token!S(data[tokenStart .. i], tokenLine, tokenCol, TokenType.normal, commentness); state = TokenState.none; commentness = false; } else if(tokenDelims.canFind(ch)) { slices ~= Token!S(data[tokenStart .. i], tokenLine, tokenCol, TokenType.normal, commentness); state = TokenState.none; } else if(tokenHeads.canFind(ch)) // < { slices ~= Token!S(data[tokenStart .. i], tokenLine, tokenCol, TokenType.normal, commentness); state = TokenState.xml; tokenStart = i; } else if(ch == tokenEnds[0]) // > { throw new ParserException(format("Unexpected token ending delimeter '%c' in line %d; at col %d", ch, line, col)); } else if(ch == tokenEnds[1]) // : { slices ~= Token!S(data[tokenStart .. i + 1], tokenLine, tokenCol, TokenType.property, commentness); state = TokenState.none; } else if(ch == lineCommentChar) // # { commentness = true; if(!includeComments) { slices ~= Token!S(data[tokenStart .. i + 1], tokenLine, tokenCol, TokenType.normal, commentness); state = TokenState.comment; } } break Lswitch; case TokenState.comment: if(lineEnds.canFind(ch)) { commentness = true; state = TokenState.none; } break Lswitch; } if(ch == '\n') { line++; col = 1; } else col++; } switch(state) { case TokenState.token: slices ~= Token!S(data[tokenStart .. $], tokenLine, tokenCol, TokenType.normal, commentness); break; case TokenState.xml: throw new ParserException(format("Reached end of file unexpectedly while parsing token \"%s\" in line %d; at col %d", data[tokenStart .. $], line, col)); default: break; } debug(LogFile) { size_t ln = 1; foreach(t; slices) { if(ln < t.line) { parserLog.write("\r\n", t.toString); ln = t.line; } else parserLog.write(t.toString); } parserLog.writeln("\r\n"); parserLog.close(); } return slices; } |
Compile time function evaluation
This will describe the basics of a very powerful feature of the D programming language: the Compile-time Function Execution (CTFE), which allows complicated functions to be fully evaluated at compile-time, irrespective of the optimization levels.
C optimizations
If you are an average C programmer, you know that simple code can be trusted to be evaluated at compile-time thanks to optimizers. For instance, if you write something like:
C-Code:
void square(int x) { return x * x; } void foo(void) { int k = square(32); /* ... */ } |
You trust your compiler to evaluate the square at compile-time, when optimizations are on. When things get more hairy:
C-Code:
int factorial(int x) { int result = x; while (--x) result *= x; return result; } void foo(void) { int k = factorial(8); /* ... */ } |
C programmers get immediately less confident about what will happen at run-time. For instance, would you say that your compiler is able to expand the above code at compile-time or not? Actually, the answer is “yes” in this particular case (unless you are using a very old compiler), but the point is still valid: this is not C code that one would write if he wants to be sure that the whole calculation be folded at compile-time.
There is also another issue: since the language does not mandate that the value is folded (and in fact, it is not folded when optimizations are disabled), you cannot create a constant out of it, such as by assigning it to a const variable.
When things get hairy
Now, let’s try with a (very naive and simple) solution of problem #1 of Project Euler:
C-Code:
#include <stdio.h> int euler1(int max) { int i, res = 0; for (i = 1; i < max; i++) { if ((i % 3) == 0 || (i % 5) == 0) res += i; } return res; } int main() { int r10 = euler1(10); int r1000 = euler1(1000); printf("%d %d\n", r10, r1000); return 0; } |
This program simply calculates the sum of all divisors of 3 or 5 below 1000. But if you look at the generated code with GCC under -O3, you will see that the actual results are not computed at compile-time, but rather calculated at runtime. I believe any average C programmer would agree that we should not expect this code to be folded at compile time.
Now, meet the equivalent D code:
D-Code:
int euler1(int max) { int i, res = 0; for (i = 1; i < max; i++) { if ((i % 3) == 0 || (i % 5) == 0) res += i; } return res; } int main() { int r10 = euler1(10); int r1000 = euler1(1000); printf("%d %d\n", r10, r1000); return 0; } |
Deja-vu? Yes, it is exactly the same, barring the initial include statement that is not required (actually, there is no preprocessor in D and modules refer to each other with the import statement, but printf is a builtin). Of course, the above example was hand-crafted to make it both valid C and D code, but being D an evolution of C, the basic syntax is the same.
Meet CTFE
And now the hattrick: in D, we can request the compiler to evaluate euler1 at compile-time by simply using the static keyword at invocation time:
D-Code:
static int r10 = euler1(10); static int r1000 = euler1(1000); |
Great, isn’t it? Now the result of the above function call are evaluated by the compiler, irrespective of the optimization levels. If the function cannot be evaluated at compile-time (usually because it has side-effects, like any kind of I/O), it will trigger a compile-time error.
We can verify that the above constants really do appear in the generated code by compiling with gdc -save-temps euler1.d and then inspecting euler1.s:
ASM-Code:
D6euler14mainFZi3r10i: .long 23 .globl _D6euler14mainFZi5r1000i .align 4 .type _D6euler14mainFZi5r1000i, @ object .size _D6euler14mainFZi5r1000i, 4 _D6euler14mainFZi5r1000i: .long 233168 .section .rodata .LC0: .string "%d %d\n" .text .globl _Dmain .type _Dmain, @ function _Dmain: .LFB3: pushq %rbp .LCFI3: movq %rsp, %rbp .LCFI4: movl _D6euler14mainFZi5r1000i(%rip), %edx movl _D6euler14mainFZi3r10i(%rip), %esi movl $.LC0, %edi movl $0, %eax call printf movl $0, %eax leave ret |
Notice how the compiler has calculated the values 23 and 233168 (respectively, results of euler1(10) and euler1(1000)) and put them in the data section of the executable.
If you are curious of what happens when the compiler cannot do the whole evaluation at compile-time, it is sufficient to stick a printf() call somewhere in the euler() function. Since printf() does some I/O, it breaks CFTE, and the compiler will happily tell you about that:
Code:
euler1.d:9: Error: cannot evaluate printf("hello, world!\x0a") at compile time euler1.d:15: Error: cannot evaluate euler1(10) at compile time euler1.d:16: Error: cannot evaluate euler1(1000) at compile time
CTFE is simple yet very powerful. The fact that it is triggered at the call site (rather than being an attribute of the function, like the inline keyword) is a very smart design choice: it makes perfectly sense for the same function to be used at both run-time and compile-time, depending on the inputs.
For the C++ guys reading, C++0x has grown a constexpr keyword that, while looking superficially similar, it is a lot less powerful, since it can only be used on very simple functions (basically, one-liners). In fact, the keyword is meant to be used while declaring a function, and not at the call-site, so it has to apply only on small functions which can be proved to always yield a constant value.
origin: Compile Time Function Execution in D
Templates, Meta-programming, mixins, and CTFE combination
Meta-programming in D is extremely powerful, yet easy to use and understand at the same time.
Writing a generic code that works with any type is dead simple:
D-Code:
T[] quickSort(T)(T[] array) //template function with type argument T { //implementation... } struct RefCounted(T : struct) { T* data; size_t rc = 0; void inc() { rc++; } void dec() { rc--; if(rc <= 0) destroy(*data); } //... } |
Imagine the possibilities:
D-Code:
import std.stdio; import std.traits; import std.conv, std.algorithm, std.array, std.string; import std.datetime; public struct XmlIgnore { public bool ignore = true; } public struct ComplexTestStruct { @(XmlIgnore(false)) int myint; @(XmlIgnore(true)) double precision; @(XmlIgnore) Duration thetime; bool truth; string[string] properties } void main() { auto data = ComplexTestStruct(50, 0.4, dur!"msecs"(100), true, ["one":"first", "two":"second"]); writeXml(data); } public void writeXml(T)(T obj, int indentLevel = 0, string name = null) { writeIndented(indentLevel, "<", name == null ? T.stringof : name, ">\n"); foreach(mem; FieldNameTuple!T) { const string mix = "obj." ~ mem; static if(!hasUDA!(mixin(mix), XmlIgnore) || (!hasUDA!(mixin(mix), XmlIgnore(true)))) { const string typeStr = typeof(mixin(mix)).stringof; auto var = mixin(mix); static if(__traits(isScalar, var)) { writeIndented(indentLevel + 1, "<", mem, ">", var, "</", mem, ">\n"); } else static if(__traits(isAssociativeArray, var)) { writeIndented(indentLevel + 1, "<", mem, ">\n"); foreach(key, val; var) writeIndented(indentLevel + 2, "<", key, ">", val, "</", key, ">\n"); writeIndented(indentLevel + 1, "</", mem, ">\n"); } else static if(isArray!(typeof(var)) && !isSomeString!(typeof(var))) { const string arrayElementTypeStr = typeof(var[0]).stringof; static if(isBuiltinType!(typeof(var[0]))) { //writeln(arrayElementTypeStr); writeIndented(indentLevel + 1, "<", mem, ">\n"); foreach(val; var) writeIndented(indentLevel + 2, "<", arrayElementTypeStr, ">", val, "</", arrayElementTypeStr, ">\n"); writeIndented(indentLevel + 1, "</", mem, ">\n"); } else { //writeln(arrayElementTypeStr); writeIndented(indentLevel + 1, "<", mem, ">\n"); foreach(val; var) writeXml(val, indentLevel + 2); writeIndented(indentLevel + 1, "</", mem, ">\n"); } } else static if(is(typeof(var) == struct) || is(typeof(var) == class) || is(typeof(var) == interface)) { writeXml(var, indentLevel + 1, mem); } else if(__traits(compiles, to!string(var))) { //writeln(typeStr); writeIndented(indentLevel + 1, "<", mem, ">", to!string(var), "</", mem, ">\n"); } else { //writeln(typeStr); writeIndented(indentLevel + 1, "<", mem, ">", var, "</", mem, ">\n"); } } } writeIndented(indentLevel, "</", name == null ? T.stringof : name, ">\n"); } private void writeIndented(T...)(int level, T text) { makeIndent(level); write(text); } private void makeIndent(int level) { for(int i = 0; i < level; i++) write(" "); } |
As this post turned out to be too long, you can ask me to elaborate on things you couldn't understand.
Have a nice day...
Ultimately, my constant dissatisfaction with the way things are becomes the driving force behind everything I do.
![[Image: sigline.png]](http://www.lf-empire.de/forum/images/unrealblack/english/sigline.png)
LF2 IDE - Advanced visual data changer featuring instant data loader
LF2 Sprite Sheet Generator - Template based sprite sheet generator based on Gad's method
![[Image: sigline.png]](http://www.lf-empire.de/forum/images/unrealblack/english/sigline.png)
There is no perfect language, but C++ is the worst.
![[Image: sigline.png]](http://www.lf-empire.de/forum/images/unrealblack/english/sigline.png)
LF2 IDE - Advanced visual data changer featuring instant data loader
LF2 Sprite Sheet Generator - Template based sprite sheet generator based on Gad's method
![[Image: sigline.png]](http://www.lf-empire.de/forum/images/unrealblack/english/sigline.png)
There is no perfect language, but C++ is the worst.