Board index » delphi » Further "Blind" challenges?

Further "Blind" challenges?


2005-06-03 03:41:18 PM
delphi176
As a solution to complaints about who used, found or invented ideas
first, and to avoid the (IMO bad) solution of having only DCUs published
for months, maybe we could reinstate more regular, short, "blind"
challenges?
I think the previous ones where kinda fun, not knowing exactly where or
what the competition is doing, nor exactly in which situation your code
would be used/benchmarked emphasized the competitive side quite a bit.
The principle would be that for 2-4 weeks, no one publishes anything,
neither DCU nor source nor even benchmark figures, and then, at the end
of the period, everything is revealed and winners designated.
Maybe we could have something regular, like a blind challenge a month?
To spice things up, and to not just optimize against known benchmarks,
we could have "unknown" benchmarks used, on which the final scoring
would be made (would help eliminate bias, if you don't know exactly in
which case you will be benched, you can not take too much risks on
case-specific optimizations... and it is a little like real-world usage:
you don't know how your functions will be used/abused).
As for creating the "unknown" benchmarks, a simple solution would be
that all participants wouldn't just enter a routine, but also a
benchmark of their creation they would keep secret too, and which they
are free to "skew" as much as they like to favour their own particular
real-world usage scenarii, only requirements being that they use the
routine (!), use no more than XXX megabytes of RAM and last approx. YYY
seconds on the RTL or a "reference" version of the routine f.i.
Eric
 
 

Re:Further "Blind" challenges?

Hi Eric,
Quote
As a solution to complaints about who used, found or invented ideas first,
and to avoid the (IMO bad) solution of having only DCUs published for
months, maybe we could reinstate more regular, short, "blind" challenges?
I like the idea of blind challenges, but I think it should run longer - say
for three months at a time. It may be too much admin work to run a blind
challenge every month. We could have 4 blind challenges per year with points
awarded at the end of each quarter. The winners are determined by adding
points for all 4 blind challenges at the end of the year. After every
quarter after the points have been awarded the source could be published so
everyone can use it in their applications.
One thing I'd like to see, however, is interim benchmarks. That way you
would at least know more or less where you stand in the competition and know
whether you can relax or must try harder :-). We could appoint someone that
we all trust to run all the benchmarks from time to time during the 3 month
period.
Regards,
Pierre
 

Re:Further "Blind" challenges?

Quote
As a solution to complaints about who used, found or invented ideas first,
and to avoid the (IMO bad) solution of having only DCUs published for
months, maybe we could reinstate more regular, short, "blind" challenges?
If this is triggered by my discussion with Pierre, please know that I'm
perfectly ready to withdraw BucketMM from the challenge or to donate my
points to people who feel they are entitled to them, if that would make us
all feel happier.
Robert
 

Re:Further "Blind" challenges?

Quote
I like the idea of blind challenges, but I think it should run longer - say
for three months at a time.
Idea of running shorter is you could have multiple blind challenges over
the course of one challenge (and not necessarrily one every month for
said challenge), final scoring could be the sum of all intermediate
challenges f.i.
I've found that challenges that don't get spotlight tend to fall behind
and be forgotten, this would be a way to "spotlight" challenges, and
keep some action going.
Eric
 

Re:Further "Blind" challenges?

It's triggered in part by that, but mainly by the observation things
have been rather 'flat' since the latest blind challenge. Sure there has
been activity and progress, but most of the evolutions took place over
months, with little "land grabbings" and overall, I don't think it was
very beneficial to the "competitive" nature of the challenges.
Having regular "spotlights" kinda sub-challenges may help keep activity
going for occasionnal contributors/vistors, would reduce the feel of
"overbearing" newcomers to a long-lasting challenge may have, and may
provide extra publicity for FastCode.
It would also probably make FastCode progress more "visible" to external
observers. As it is, progress is happening on various challenges, but
unless you're a regular follower, it is hard to know what's happening
exactly, f.i. is Lower_Case_JOH_SSE_78b an improvement? How does it
stands relatively to others? What is BucketMem_RH_775 bringing to the
table? What are the algorithms/strategies used? etc.
All kind of things you can find by yourself sure, but also all can of
things that are usually discussed in a much more accessible manner (for
outsiders) at the end of blind challenges ;)
(and come to think of it, there is quite a jargon specific to the BASM
project which if clear and useful for insiders, probably scares away
outsiders)
Eric
 

Re:Further "Blind" challenges?

Hi Robert,
Quote
If this is triggered by my discussion with Pierre, please know that I'm
perfectly ready to withdraw BucketMM from the challenge or to donate my
points to people who feel they are entitled to them, if that would make us
all feel happier.
That's not what I want at all. As I said in a previous post, what's in the
past is in the past.
All I want is reassurance that all my hard work and hours of research is not
going to be copied the moment I release FastMM4. Programming is quick and
easy, coming up with new and better strategies is the hard part.
Regards,
Pierre
 

Re:Further "Blind" challenges?

Quote
All I want is reassurance that all my hard work and hours of research is not
going to be copied the moment I release FastMM4. Programming is quick and
easy, coming up with new and better strategies is the hard part.
You are right. But in that case: protect it with a better license!
I would not be surprised if some of the optimizations (on known algoritms!) have relevance for more than just Delphi, or even more than just 80X86 platforms.
And I know for sure that people outside these environments are reading this particular newgroup only for the MM challenge.
I am impressed with some of the results I can achieve by using the current top three, both at work, in a heavily loaded web environment as in my hobby, audio dsp, which would be a very good benchmark by the way: fft's and large buffers, many (re-)allocations, behaving almost just the opposite of strings)
That said, what the team has come up with is impressive and certainly for outsiders it has always looked like very much a team effort: bouncing off each others code is a good thing!
And some of the challenges here prove just that.
 

Re:Further "Blind" challenges?

Hello Pierre,
Quote
All I want is reassurance that all my hard work and hours of research is
not going to be copied the moment I release FastMM4.
Aren't your comments slightly biased?
When you use a feature in FastMM that appeared previously in BucketMM you
call it "obvious", "logical", "trivial", "RTLMM did that ten years ago".
When I use a feature in BucketMM that appeared previously in FastMM you call
it "copy of hard work and hours of research".
lol,
Robert
 

Re:Further "Blind" challenges?

Hi Thaddy,
Quote
You are right. But in that case: protect it with a better license!
I'm afraid the only way to protect it would be to patent the algorithm, but
that is not something I'd want (or can afford) to do. Copyright doesn't
prevent others from copying your ideas, only your actual implementation of
those ideas.
Regards,
Pierre
 

Re:Further "Blind" challenges?

Hi Robert,
Quote
Aren't your comments slightly biased?
You seem to think so, but I don't. Doesn't matter... everyone is entitled to
their own opinion.
Quote
When you use a feature in FastMM that appeared previously in BucketMM you
call it "obvious", "logical", "trivial", "RTLMM did that ten years ago".
I call a spade a spade. Sorry if I hurt your feelings.
Quote
When I use a feature in BucketMM that appeared previously in FastMM you
call
it "copy of hard work and hours of research".
No I didn't - read the sentence again. I am talking about *FastMM4* here,
not of older versions or what happened in the past. FastMM4 probably took me
longer to develop than FastMM2 and 3 put together. When you see it you'll
understand why.
You haven't even seen FastMM4 yet, so how can I accuse you of copying it?
Regards,
Pierre
 

Re:Further "Blind" challenges?

Quote
>You are right. But in that case: protect it with a better license!

I'm afraid the only way to protect it would be to patent the algorithm,
but
that is not something I'd want (or can afford) to do. Copyright
doesn't
prevent others from copying your ideas, only your actual implementation of
those ideas.
But you can set the terms for the license, things like:
- You can not claim ownership of my ideas, and any derivative work must give
credit to FastMM.
- Branching of FastMM for your own derivative works is strictly prohibited
without prior written consent.
- For the free usage of FastMM, any modifications/improvements made to
FastMM are to be shared with myself.
- If you plan on selling a derivative commercial version of FastMM, special
arrangements and terms can be made.
- Closed source applications must give credit to FastMM either on the
product or in advertising.
- This copywrite license can't be removed from the source.
There are ways to be fair and open, and still protect your rights. The above
is just a couple examples of things I have seen in other licenses. If someone
doesn't like the terms they could always pay you to get a special license.
DD
 

Re:Further "Blind" challenges?

Maybe this has been discussed or proposed already, but couldn't a version of
the B&V be created, posted, and each MM author could then compile it with
his MM, all executables being submitted in a package for download?
 

Re:Further "Blind" challenges?

That is exactly the point I tried to make.
Patents aren't very well suited to protect intellectual property worldwide (see the discussions in europe). Licenses are. If you study the various main open source licenses you will understand they do just what you want. A combination of licenses is often given and a user can often make a choice. That includes, btw, a combination of open source and commercial licensing. It is for you to choose what you think is proper. I don't think any of your fellow competitors is after your intellectual property and I am shure, as they already stated, they would give credits where credits are due.
An example:
If an open source software is vital to my product, but the combination of my closed source part and the open source part would conflict with my commercial software as a whole, then - provided it is offered - I'd always opt for a commercial license for your code.