[»] your solution
by albert fuller - Dec 4th 2002 10:37:36

you have a great idea and I like much of what you have to say.

I would just like to add my thoughts on the design level: I feel you need to confront the variability of Linux distros and file layout. The question is how you can add an app and a definition of its configuration elements (even with choice lists if possible) in a straight forward way to this system; and at the same time your system could support multiple states based on machine.

It's no good for the user to effectively endup using your tool as a blank scheme where user has to add in new objects (eg., mySpiffProg), in need of configuration, and the definition specifics to get some configuration interface; and then configure the object. At this point, just go straight to configuring the object arbitrarily.

The issue I feel is one of developing a model of distributed knowledge (that's essentially what a Linux-universal config tool would be) where your tool is the implementation of that model.

Now this model must do 2 things: (1) seperate knowledge requirements from utilization and (2) seperate knowledge requirements from knowledge need.

This model can exist as a public online database of configurations, and your tool if it could get/create/poll the current environment to determine all available objects and then query your public db to download the required configurations. Any requests for configurations that the public db did not have would be logged for later resolution.

Possibly users who had a configuration to submit to the public db, could do so through your tool which would then accept item to be validated before going into distribution.

example stop:
I have just installed your-config-tool; it polls my system and determines there are 3000 configurable objects (ok I run SuSE Linux) -- your-config-tool would need to determine which configurable objects are already properly configured and which are not -- then I could download those configs I had selected from the picklist of objects in need of configuration. I would then get those selected configurations loaded in my machine and what was not available in the db was logged by the db for future acquisition. I even get a chance to have you guys poll my system for configs that you are in need of (as in thousands of thousands of people want my my freeware program mySpiffProg dispite its tricky config setup) and/or one I directly submit.

Now, I jsut got me a reved-up bash or emacs configuration along with a nice config for wine, apache and squid (all version relevant of course). Your db uploaded 3 configs (to by polling my system and one by user submission - I just want to get mySpiffProg out there!)

********

Ok, you see the point I am trying to make. I see the issue essentially one of information distribution in a distributed community where users can add to the pool of configurations as well as get configurations a la carte... This is what I think addresses your idea of "universal configuration." "IT'S NOT JUST A TOOL, BUT A WAY TO linux-BLISS)

I have posed my take on the issue at the widest level, to see what the basic interaction could be like given your "universalization" on the config theme

have fun
and good luck

[reply] [top]

[»] API, API, API!
by AstroMan - Nov 22nd 2002 20:07:37

This system sounds great. As a developer, the only way I see this "catching on" is if there is a simple api. The system <b>needs</b> to be applicaiton oriented and not system oriented. I am not fond of the windows registry, however, I do like the fact that it is applicatioin oriented. One application can go in and save it's configuration data without affecting any other program. Most importantly, there is no need for a user to know it is there. If a "power user" wants to get in and tweak, well then let them get in an tweak. There should be no reason for a "normal" home user to have to edit config files, registries, xml files, etc to use a program. When I write software for Linux or windows, I try to make them need no user intervention to setup. Linux needs to take this approach as well. One good selling point for your project will be if you can come up with a simple api, that will be a breeze for a programmer to store their settings in. If a certain apps needs more complicated settings, then allow that to be possible through an api, but don't make it the only api. If I have an app that only needs to store a string, then I want a simple call like<br>
save_string_setting("MyApp", MYVERSION, "key name", "some value");

Then at program startup it should be something as simple as<br>

char *myValue;
myValue = get_string_setting("MyApp", MYVERSION, "key name");

Notice that memory allocation would best be handled by the API and at program termination a call to cleanup() would free any memory. I could live with managing my own memory thou.

Best of luck.

[reply] [top]

[»] Why this can't work.
by David J. Sankel - Nov 22nd 2002 10:33:29

The Law of Leaky Abstractions applies here. Unix will always be fundamentally limited by its complexity. A system like the one presented is bound to break. The current unix way of doing things is every program has a different kind of configuration file and the format of the file may change between versions. The maintainters of this system must update the versions of their sofware every time one of the things it configures for changes.

In turn, this project would not really help any sysadmin. Anyone who runs djbdns better know the configuration format, anyone who runs qmail better understand the /var/qmail heirchy. The system presented is bound to be leaky so an understanding of the underlying configuration files is required.

When K&R and friends invented UNIX, they invented a language for it called C. Neither C nor the UNIX libraries had special facilities for configuration files, just files in general. If someone was to change this paradigm, the system would no longer be UNIX and it wouldn't be, in any decent way, compatible with UNIX.

So, yes, I think this idea couldn't work in any practicle way. What it does show is a fundamental limitation to UNIX that new operating system designers should take into account.

[reply] [top]

[»] Re: Why this can't work.
by smithdev - Dec 5th 2002 16:55:00

I agree with what you say. I was going to take a more database-ish stance, and point out that what we need to do here is protect the information from the administrator and the administrator from the information.
It might be fun to write a Postgres database that knows how to register applications, their configuration file locations, and abstracts their formats to the point you could hit them with SQL (!?!)
Such a hypothetical implementation/maintenance nightmare, while instructive as an exercise, would fail the admin-panic test: you very likely couldn't boot from a floppy in an emergency and get anything useful out of it.
There just aren't any silver bullets. Have we considered the LSB? Perhaps this project could be included in the LSB so that the community will be more likely to migrate towards it.

--
gentoo is all that

[reply] [top]

---

[»] Re: Why this can't work.
by submissions - Dec 27th 2002 23:18:45


> The Law of Leaky Abstractions applies
> here. Unix will always be fundamentally
> limited by its complexity. A system
> like the one presented is bound to
> break. The current unix way of doing
> things is every program has a different
> kind of configuration file and the
> format of the file may change between
> versions. The maintainters of this
> system must update the versions of their
> sofware every time one of the things it
> configures for changes.
>
I read the URL and I can't see any connection between the examples of Leaky Abstractions. Except "the world is not as perfect as you thought, oh well...". Anyway, I have to point that runlevels and init scripts are optional. They are the good way for a sysadmin will manage his server, but on the other hand linux can be as simple as AUTOEXEC.BAT CONFIG.SYS. And even simpler.

There is no complexity in UNIX. Again: UNIX is terribly simple. The current configuration systems are hacks by sysadmins.

[reply] [top]

---

[»] Scope of project
by venomx - Nov 14th 2002 19:14:21

while I have to agree that standard configuration is needed in linux, I think that the scope of the project is too broad. IMHO a much better approach is to attempt to create a standard set of tools (library) that handles parameters and values indexed by key. While there are certainly complex configuration formats in applications, I believe that they are the exception rather than the rule.

If a sufficient library were created that offered 80-90% of the needs of programmers it would simplify the configuration of all of those applications that don't require advanced configuration formats. I also believe that a simple key, parameter, value format can be adapted to more advanced files albiet with a little work. This should be the job of the application to handle the various complexities that arise such as parameter contexts and overrides, etc.

The simpler the better. I believe that once a good toolset is in place people will want to use it. Key, parameter, values models data structures accurately. A key item that offers flexibility is the muli-valued parameter using a delimiter. Quoted strings should be used to enable delimiters to be used in a config:

[ KEY ]
param1 = something, "something else, blah,blah", another

Hierarchies can be created using dot notation:

[Global]
param1 = something
param2 = something, "something else, something else,", one more thing

[ Some local key ]
param1 = foo
param2 = bar
param3 = baz

[ Some local key.overrides ]
param1 = something different

Etc. So contexts are created using dot notation rather than braces. Each level in the heirarchy that is associated with a given key is up to the application to handle. After all the config file should just store parameters and values. If we need to handle default values, we can add another section:

[ Some key.defaults ]
param1 = default foo
param2 = default bar
param3 = default baz

Comments can be stored easily and associated with either some key or parameter. This way the documentation is in the config file. If for instance a GUI were to be made, it can simply pull the comments out of the config file and display them.
For similar generic items you can create a key that using numeric increments like for modelines in a xconfig file.

[ Modelines ]
total modes = 4
mode1= ...
mode2= ...
mode3= ...
mode4= ...

The toolset should not need to be aware of this situation, the application should interpret each modeline by
I just don't think the problem requires massive amounts of engineering. This solution is simple, it works and can be adapted. If we find out for some reason that a particular requires some complex format that can't be adapted (I refuse to believe that with enough creativity a config format can't be adapted somehow), then extend the format in an intuitive way to allow more advanced configuration. This way you can get 80-90% coverage for the simple config files and still allow advanced functionality using the same library.

I believe that this is the key to a uniform config system. Once everyone is using the same library, then you can create a generic framework to localize configuration of all these various config files by extending the toolset to include a simple registration system allowing a single app to version configurations, save, backup, alter, use ACL's etc. But the first step is to "market" this flexible toolset to developers.

Start small, why spend massive amounts of time engineering something when this solution is so trivially simple it's possible in such a shorter time frame. Once this phase is accomplished, you can create another toolset (library) that will parse the config files and build a GUI automatically by reading all keys, creating the menu hierarchy from the dot notation, reading all parameters and putthing them into a GUI list control, extracting the comments, and anything with a *.default can be plugged in. Heck you could even create an *.option if you needed in order to dynamically generate all of the possible values for a specific parameter. This can all be done in various languages with ease. A C API could be created, a set of simple C++ classes, perl modules, python modules. Porting something like this is far easier than some 3 tier system with XML and all that jazz.

I personally would rather spend my time on something like this rather than some massive engineering effort to try to unify already existing config files. This is just much simpler than a 3 tier approach. Just my $1.50 here :)

[reply] [top]

[»] a GUI using GNOME or KDE ???????
by Kloss Korban - Nov 13th 2002 15:44:40

Or a better idea might be to create the gui in a toolkit like qt or gtk, and not a "desktop enviroment/application suite" like Gnome and KDE.
An even better idea would be to staticly compile a fltk gui.
But if you did that, you would never expirience the fun of not being able to configure your system because of gnome-scrollkeeper errors. Yah, I know if that happens you can still just edit the text file, YUP! thats right. But thats not fair, The Newbie, WinIdiots can't handle that, they might have to read a manual instead of clicking on things until it works.
oh, if you need a gui config. Webmin rocks!

[reply] [top]

[»] Some Suggestions: Precursors & Parentheses
by Richard Russo - Nov 4th 2002 08:28:23

I'm coming to this discussion late, sorry if I restate someone else's opinions.

I'd like to point out that there are a lot of other systems with similar goals, some very successful. Take webmin (www.webmin.com), for example. It has a wide range of plugins which add to the list of programs that webmin can configure. Its UI is a webpage, served over https by an internal server. HTML is nice for developing the kinds of UIs needed by this program (forms), so new forms are developed quickly.

Where webmin is practical, the next example is theoretically nice: Emacs M-x customize. Emacs customization groups provide a general, form-based way to configure Emacs extensions. The documentation, including samples, is right there in front of you. I'd say that if a new Emacs user could figure out how to get into the customization groups, there'd be no problem configuring the favorite extensions. This system has some things that you aren't assuming, however, like some standard ways in which all programs read in their configuration (through emacs-lisp variables).

I really respect this project, I think it's a great idea. Check out these systems so that you could possibly incorporate their good points.

btw, I saw in the author info that you were at the Honolulu ICPC? I think I remember seeing Messiah on the roster. I competed with the team from UCF there.

--
Watashi wa tamago no uchi de gyuunyuu o nomimasu.

[reply] [top]

[»] importance of a standardised data structure
by Hans.v.Buitenen - Nov 3rd 2002 16:07:08

Introduction

This doc is a reply on http://freshmeat.net/articles/view/565/. Please note that i recognize and agree on the concerns that you have on the linux way of configuration.

In short i would like to express the importance of a standardised data structure. Also i think it is very impotant to contantly remind program devellopers to avoid unneeded complexity.

I hope you understand my point below and find it a positive contribution.

In my percetion a configuration consists of three things:

About the data structure: We've seen two way's of not doing things, the windows registry and the linux way. Why are these a screw-up? Because the lack of a standardised structure. If you have a standardised data structure then your'e halfway, and, it doesn't seem that hard to do. Also it is not really importand how you do it, as long as it is standardised everybody can find whatever they need.

Let me explain this. First you design a structure. The data has to be seperated into levels. f.i.

Root, Vendor, Application, Data Group, Data Item, Value.

By defining all elements of the data structure, app designers don't need to "invent" or interpret standards for them selves.

As soon as the data structure is clear you can start thinking about the implementation of the data structure. (Design and implementation are relativily independend). The data structure can be implemented using several techniques:

The internal structure should be standised as detailed as possible and have freely available access tools.

The database implementation f.i. could be mysql, a lichtweight, reliable and fast database. Discourage the use of "a different kind of database for each app", this really sucks. Even if mysql is to big for you, it will not be to big if you add-up a few apps. Other databases may be just as good. Important is that all apps use the same one whenever possible.

It is very well possible to mix these techniques, however, a few things must be kept in mind:

An example: A vendor called "AVendor" creates a program called "XApp". He could choose to do the following:

/etc/AVendor/XApp/XApp.conf

/etc/AVendor/shared/X.conf

/home/bob/.etc/AVendor/XApp/XApp.sav

or

/etc/AVendor/XApp.xml

/etc/AVendor/shared.xml

/home/bob/.etc/AVendor/XApp.xml

Suppose a certain vendor has a complex suite of applications and wishes to use a database. Then you could get:

/etc/AVendor.database

/home/bob/.etc/AVendor/XApp.xml

Vendor is used to be able to group certain applications. Examples of vendors are:

A pitfall in choosing a vendor name is that f.i. redhad would put every app of his distribution in /etc/redhat. If someone later on dowloads an app then it would go in /etc/gnu. So i think that the data location should be determined by the app writer and not changed by a certain distribution.

Only proper standardisation and documentation can prevent misinterpretation of the data structure design.

What to do with os dependent settings. It might be usefull to give guidelines for this. Maybe keep the data separate, use standardised notation and translation.

The user interface is a complex topic. Normally i would not say that the userinterface should be a representation of the data structure. In this case i think we should try to let the two resemble as much a possible.

The configuration file datastructure should be a logical one that is easily understood. Not just a bunch of names and values in random order. In fact, these files are a (simple) user interface, otherwise they would be binary.

What has been said about the data structure is also valid for the user interface. It is not important how you implement it, it is important that the way of using it is standardised. So if you see the data files themselve as the simpelest user interface it is logical that all "higher" user interfaces sould have the same structure but are easyer to use.

Ehh... I don't have anything to say about this so I won't.

I note that you are clearly focussing on a certain app while my focus is more to standardising the data and way of working. This is an important difference. I think a good app to manage the settings is a good thing but then again you may be overtaken left and right by others such as linuxconf etc. I think it is more important right now to get the basis right. The app you have in mind could be serving as a test to see if the data structure is choosen right.

Also you are focussing on the use of text based config files. This is not in conflict with what i write, I even think this is good whenever this solves the problem, but my view goes further. If someone decdes he needs xml or a database then you've lost him. My proposal encourages to keep following the standard and use the same logics whithin those techniques and makes sure everyone can access this data.

The three tier architecure.

I think it is good to keep this topic in mind but having a widely accepted, standardized data structure is the first thing to worry about.

You directory structure is flatter than what i propose. I think my structure has an advantage towards commercial software vendors. I think these should also enbrace our proposals.

Further, you directory structure uses "deamons". I have learned that unix has only files and processes. A deamon is nothing more than a proces running in the background. Many of them can be executed as a normal app. I would try to match closely to the Unix philosophy and not try to determine a border between programs and deamons like windows does. You might get burned if you do.

Configuration locations are not fixed: Arrrrrrrggh.... Of course they should be. The app should be able to read from other locations but there is a standard location that all Linux distrubutions use. If you deviate from that you sould better have a clear reason for it.

Forms, Wizards

No, I don't agree, the Windows registry is a complete mess that has partially been deliberately been made unreadable. This database has been written with comercial motives. I repeat, this is a screwup because of the lack of clearity towards its users. Of course, remarks in files are a big advantage but you can do that in a database too (the forms you mention) so there is no real point in discussing the implementation technique other than complexity and performance.

Wizards

To me the main task of a wizzard is

Please evaluate each wizzard on these rules. After using a wizzard you should get a working situation.

Metadata

The data should be based on standards. Be ware of the microsoft effect. Every dummy is going to add metadata to his setup. Look at what they do with the registry, every stupid program that needs 5 lines of data to init has reg entries and needs to reboot.

We have 5 lines of defence against complexity

Rule 1: Keep everyting as simple as possible, but not simpeler. (That quote's not mine)

Rule 2: Clear data structures

Rule 3: Logical configuration

Rule 4: Documentation

Rule 5: Documentation (Also document the setup data)

You have to have a real complex program to need to use metadata. Use it only when this kind of complexity is needed. Many things can be done with straight forward configuration and documentation.

Access Control

Access Control should be clear. The current linux system puts user accessable data in the home dir and system data in /etc. This is straightforward and simple.

Doing access control within one configuration area or even in one file wil cause more complexity so avoid making it fancy. If you need to go away from the linux file access rights then you end up with a database solution with rule based access. This confilcs with "Everybody should be able to use it" There's no such thing with rule based access controll.

Yes, redo the linux config but start at the basis. Standardise the data, then go to the app layer. Keep in mind that the data usually lives longer than an app.

Be ware of inviting complexity. People tend to use what you offer them. I think there should be a clear statement about avoiding unneeded complexity and a few samples of how to configure simple and less simple programs.

Greetings, Hans van Buitenen

[reply] [top]

[»] COAS
by N.M. Holtz - Oct 29th 2002 06:21:55

Caldera made a start at something like this several years ago with COAS (www.coas.org). I think the last time anything was done on this was late 1999. Might be worth looking at, as there are some technical notes (and code) there. Also might be useful to find out why it didn't fly (I suspect it was the competition from linuxconf and webmin as much as anything).

Interested me as Python was to be the source language, which sounds like a good idea.

[reply] [top]

[»] Has anyone looked at NetMAX?
by burple6 - Oct 24th 2002 08:58:09

Hi, I wanted to mention that there is a commercial product that does a pretty good job at most of the things discussed in this article.
The product is called NetMAX and is a web-based administration frontend to a Red Hat-based distro.
Though some of the things on the list are not currently implemented, I am sure it would be a trivial thing to add them if there was an interest.
Anyway, thought I would mention it.

[reply] [top]

[»] Looking forward to this - My 2 cents worth
by Andrew Fowler - Oct 22nd 2002 13:41:35

Anybody still reading this ?? Anyway.

To answer your first question:
Is this something worth striving for? In my opinion, most definitely yes.

A few comments on the spec so far: As is already obvious from some of the feedback, not everybody thinks newbies should be allowed to use Linux. Some appear to suggest the XFree is a complete waste of time .... I'm definatly not one of them.
From my point of view, as a developer, there are some apps which I toy arround with and reconfigure weekly. I know the config files for these inside out. Then there are those apps and daemons which get configured once a year when I feel I need to change something. I definatly need all the help I can get with these. What I mean is: there is no such thing as new user/advanced user. There are just the apps you know, and there are those you don't. Maybe rewording the spec to this effect may help avoid some of the flaming that's been going on (and undoubtedly will carry on). no suggestion at the moment.

As far as features I'd like to see:
- on the GUI end (or for that matter from the command line), built in documentation for all fields.
- on the back end: ability to specify validation info. Ideally this would allow not only simple field validation but also relational validation (a=1 therefore b > 6) !!! That's going to be fun
- ability to export/import subtrees (purpose: to take the config info over to another machine, to be able to play around with certain elements and then restore them to previous state - without afecting any other changes made in the mean time, or even to allow the config section to be posted on a mailing list when asking for help)
- the ability for user to configure their own application preferences (i.e. the dot files which are in the user's home directory) - not sure if this is explicitly included in the draft.

[reply] [top]

[»] API/Library
by Grahame Jordan - Oct 20th 2002 01:33:58

I believe that for this project to succeed a well defined configuration API/Library needs to be created for as many languages as possible. For a start we would target:
C
C++
bash and other shells
perl
php
java
KDE
Gnome
tk
and any other languages we can think of. Then one by one in cooperation with each project bring them over to the config libraries. I admit that this may be asking a lot from some projects to cooperate but I believe most will. Maybe a translation API for to cater for the odd project that does not wish to use the new API.

It may be necessary to have dual formatted conf files in the interim. One that the legacy program reads and one that uses the API with a config parser to write it out as the legacy config file. This is not desirable but until the config library is ported to that particular program it wiull suffice as an interim measure.

Today when writing a program it is required to write an entire parser to read and verify config files. With the library already built it then becomes a few simple calls to it. Required paramers, dependancies, options etc,etc are all handled easily and consistantly with the new libraries.

The exact format of the configuration files is yet to be decided. Some cry XML here and others RDP there. This must be decided first on the merits and carry on from there. Personally it does not matter to me what is used, but that it is agreed upon soon. I would hope that once a discussion specifically for deciding the file format is completed that we can all move to the next stage. (It really does not matter what we use as long as it will do the job). Anyone follow the unixconfig mailing list. That is where it died. Come on if we can agree on this quicker we can progress quicker.

Once the file format is decided a clear and precise definition for types, dependancies, requirements, etc. needs to be formally documented and agreed upon.

OK, now write the Libraries for each of the languages that wish to be supported. The API should be the same for each language (as it can be). Write reference code and examples for each language with good documentation (Now there's another ....), and testing etc.

We then can target certain projects and cooperate in the rewrite of the configuration sections of code to use the standard libraries that we have provided.

Then dependacies between programs, global variables such as (HOSTNAME) can be accessed by all programs. etc, etc.

This can extend to enterprise management systems.

The GUI bit should then be the easy part. But it is important that the GUI parts are developed along the way in several reference languages so that on release date we have a product.

It would be great to have the configuration system define the GUI components and layout without having to write the code to do it. The easier this is the better. The configuration can define the checkboxes, textfields, comboboxes, tables, help, dependancies, requires, ordering tabs, wizards, etc, etc. There should be little need on the GUI end to write a great deal of code. We should also include a cursor driven configuration system or even a purely text based configurator. It is all in the API for each situation.

This project will need the cooperation and support from many areas of the community. It will require acceptance of decisions that are made on merit. There is definately more than one way to do this. More important is that we use a standard. Maybe in the future some will wonder why we did it this way or that but as long as the standard is adhered to there is no big deal in changing it.

It would be great if members of other projects joined this one and shared there wealth of knowledge. This project is necessary for unix to go ahead.

Lets do it!

[reply] [top]

[»] Plugins
by kop - Oct 16th 2002 15:06:55

The definition of what a certain config-file format looks like (referred to in the article as metadata) should (naturally) be stored in separate files for each, so that people could create and distribute these format definitions in a simple way.

I'd say that for a system like this to ever grow popular, it needs to be heavily based on the "plugin view" in general. Creating new plugs/defs should require no awkward object file compilation etc.

On projects like this, BLOAT is alway waiting just around the corner; A good design would enable features such as "History Log" thorough a simple RCS-plugin.

[reply] [top]

[»] Worthy Project, but for whom?
by Spin - Oct 2nd 2002 11:44:36

Configuring anything (computer, car, bookshelf, etc) can be a difficult and time consuming and may require expertise in the applicable field. However, how often do you reconfigure you Linux box? Obviously this answer varies enourmously.

For the home hobbist, it may be hourly and the convience of remote GUI is a godsend. For others, the inconvience of once a week/month/year going and logging on directly to administer a change is a small price to pay for the lack of daily security headaches.

The issues of configuration and ongoing maintenance should already be solved long before a system is implemented into a "production" environment, in most cases it probably boils down to 'change this line in this file here' as the only modification ever done on an ongoing basis, despite the fact that the machine itself may have several hundred customised/standard configuration files.

I'm lucky enough to use Linux for work and play, but my approach to configuring it for work and play is completely different.

[reply] [top]

[»] The Dotfile Generator
by Jacob Sparre Andersen - Oct 2nd 2002 10:19:58

Have you looked at how the Dotfile Generator attempts to solve the same problems?

Have you considered how to handle configuration file formats that are close to complete programming languages?

[reply] [top]

[»] Data
by Robert Trebula - Oct 1st 2002 10:04:23

Though I'm still quite skeptical about a real need for such a system, I'm going to add a few points to the discussion. And maybe once the succcess of the project will beat my skepticism :)

The functional and architectual model (here and on the project website) look good to me. I just miss a data-design.

What will be the structure of data flowing through the system? You only speak about text fields, integers and such non-structured entities. But in real life and in real configuration schemes, the information in configuration files is more structured.

Look at apache for example. The configuration options like DocumentRoot or ErrorLog can be specified for the whole server, for one virtual server, for one directory etc. Parameters in subdirectories are inherited from their parent directories. Some configuretion options are not allowed or have no meaning for directories and can only be specified globally. Values of some configuration options include other configuration options... I guess you see the point.

Another look at the data: I propose the low-level data entity should be a chunk of a diff file to the orignal configuration file. This approach has proved to fit for the need of editing files by various players - the user editing the configuration file directly, your system or maybe even some other configuration system used on the machine. The main pros - consistency (it's harder to damage a file with a diff than by rewriting it from scratch), auditing (when you log a diff-file, then you can easily see who did what and why).

And by precisely defining data-structure, you can precisely define interfaces. A great pro for a modular system like yours.

Luckily, XML is very flexible for use with structured data, so there should be no problem with the implementation.

Hope this brings something for you,

R.

[reply] [top]

[»] Re: Data
by Justin Yackoski - Oct 1st 2002 15:51:51

The actual XML data is organized by nested <section> tags, which accomplishes the necessary grouping. The individual parameter/value pairs are inside one or more levels of <section> tags. Currently, we don't have a parser for apache style config files, but this would be a goal. Even with INI style files (like samba), the same issues rear their head. For example the guest ok directive can be used to globally allow guest access to samba shares, and any sections inherit this default value unless it is explicitly over-ridden. From my experience, apache works in the same way (more or less).

At the very least, we will need to provide a way for the system to be aware of default values, which will be inherited by all sections including the highest or global section. If carefully written, this inheritance could easily be extended & handled as the XML is processed (either to or from the top-level) by the middle layer. It is going to be a tough problem to tackle, but as you mentioned, I think the nature of XML will help make it easier. Properly handling defaults & inheritance will be very important for the project's success since otherwise it won't be very useful for a lot of situations.

As far as using only diffs of the new and current config files, we can currently completely reconstruct the config file *exactly* (byte for byte identical) as it was from our XML representation of the data. If we can do this, and we checked to make sure the file hadn't been changed by one user as another was editing it by hand, we should be ok. Either way, since we can re-create the exact (or slightly modified) config file, producing a diff of the two wouldn't be difficult either, and may be a good option (among others) for logging. Thanks for your input!

[reply] [top]

---

[»] You gotta laugh!
by Ged - Sep 30th 2002 17:02:41

Some people say it is impossible; some say it already exists.

I'll just say this, the article gives the goals:
The system should be easy enough [for] new Linux users.
The system should be flexible and powerful enough [for] advanced users.

I don't think there is any existing solution that meets just those two goals. I don't think meeting them should be impossible (though it will be hard). And I do think doing this would be fantastic for the GNU/Linux community. Best of luck!

Ged

[reply] [top]

[»] GConf
by Nils O. Selåsdal - Sep 30th 2002 10:15:48

Seems like this article describe GConf. Widely used in Gnome, but not gnome dependant.

[reply] [top]

[»] Re: GConf
by Justin Yackoski - Sep 30th 2002 23:45:11

We did look at GConf among other projects, hoping that we could build on some existing code. One of GConf's main flaws is that the method for adding support for a new thing to configure involves writing a lot of C code and various other things, which we felt was too cumbersome.

We want to make things simpler, and the architecture for GConf and most of the other tools wasn't flexible enough to accomplish what we're looking to do. Unfortunately (depending on how you look at it), it looks like we'll end up writing lots of new code in order to accomplish this. The upside is that since we're putting tons of effort into planning, soliciting comments, and really thinking this through, the initial release should turn out pretty close to what we're going for.

[reply] [top]

---

[»] Add support for integrity checkers + script generation
by Fremar - Sep 30th 2002 05:42:35

I certainly hope you will pull it off to devellop such a project.

And even though you have already burried yourself under an enourmous workload I want to add a request: build in some form of support for integrity checkers (e.g. tripwire). One thing which has kept my from using any of the existing tools like linuxconf or webmin is that I always get some tripwire error messages about changed files. Of course, checking the logs will usually give a clue as to why some configuration file has been altered, but clues are not enough where system integrety and security is concerned, I want an exact listing of the files that have been altered.
This might even be implemented in the form of an additional backend which takes the list of modified files and feeds it to the tool that recalculated the checksums and updates the database; all with proper authentication of course.

Now that I think of it, I can add a 2nd request right now: might it be possible to have gui or web frontends generate a list of commands to have the command-line tools re-create the same change? This would be ideal to automate configuration changes for clusters or groups of similar workstations: configure a service like samba on one of them through the gui and have a script to make the same changes on all the other machines. In my optinion this will be more powerful than a centralized configuration solution since the same script can also be applied to new installations (e.g.a post-install script for a kickstart installation).

[reply] [top]

[»] Re: Add support for integrity checkers + script generation
by Justin Yackoski - Oct 1st 2002 00:19:32

Integration with tripwire/aide/etc is certainly not a pressing goal, but it is a great idea. We're planning to have the middle layer log changes in some way(including what files were changed) and there are a lot of good reasons to make optional components available which do other things with config changes (modification detection, copying to multiple boxes, creating a script to get from state A to state B, etc.). To do things like managing users, changing hard drive settings, etc. it may turn out that this type of change to set of commands translation is necessary.

You have some good ideas, and we will try to make it so that even if we aren't able to implement and stabilize all the requested features, it will be relatively easy for people to add them later since our system will be as modular as possible.

[reply] [top]

---

[»] Make it easy for lazy people? Nahhh!
by Guillermo Sobalvarro - Sep 29th 2002 09:40:11

Use FreeBSD as an example and take it from there. This is for people who know the insides of their systems and not for superficial types who just wan't a slam-bang-thank-you-ma'm OS. FreeBSD requires a "relationship", a commitment, and not just a one night stand. This is not for lazy asses. The market wants people who know what they are doing, not people who know how to use a GUI. If you want dancing clips, Einstein and doggies, stick to XP! Guess that's what all the envy is about.

[reply] [top]

[»] Re: Make it easy for lazy people? Nahhh!
by Bryan Evege - Sep 29th 2002 22:52:51

Attitudes like this is what is holding Linux & Unix back from the mainstream desktop. Most people (like me) don't have the time it takes to become a unix/linux guru. It's not about being lazy or dumb it's about getting a job done as quickly and efficiently as possible. This just isn't the case for most things in linux/Unix. I don't want a relationship with my computer. I want it to be easy to manage and work reliably. Preferably in hours and not days or weeks.

I'm no Microsoft lover but programs like this (and some reasonable documentation) will only help Linux/Unix become more mainstream. For those of us who have better things to do than have relationships with our computers that is. Keep up the good work. And btw, please write some good documentation.

Unix/Linux Newbie


> Use FreeBSD as an example and take it
> from there. This is for people who know
> the insides of their systems and not for
> superficial types who just wan't a
> slam-bang-thank-you-ma'm OS. FreeBSD
> requires a "relationship", a
> commitment, and not just a one night
> stand. This is not for lazy asses.
> The market wants people who know what
> they are doing, not people who know how
> to use a GUI. If you want dancing
> clips, Einstein and doggies, stick to
> XP! Guess that's what all the envy is
> about.


[reply] [top]

---

[»] Re: Make it easy for lazy people? Nahhh!
by Juha-Mikko Ahonen - Oct 2nd 2002 01:16:31


> Attitudes like this is what is holding
> Linux & Unix back from the mainstream
> desktop. Most people (like me) don't
> have the time it takes to become a
> unix/linux guru. It's not about being
> lazy or dumb it's about getting a job
> done as quickly and efficiently as
> possible. This just isn't the case for
> most things in linux/Unix. I don't want
> a relationship with my computer. I want
> it to be easy to manage and work
> reliably. Preferably in hours and not
> days or weeks.
That's not the way to do it. Blaming others from "keeping Linux back" just because you don't want to commit to making an "easier" system possible. If you don't have time for Linux, quit using it. It is not for the weak of heart.

And why should Linux be put to the mainstream desktop? Home users have difficulties with even Windows. Who will help them when they're trying to modify their Linux installation? Corporate desktop is another matter. Corporations have system administrators who will do the necessary configuration for you.

But if you fear to throw yourself into a relationship, consider some alternative. Linux needs the commitment -- otherwise how would you manage when you screw up your system. Reinstall? And do all the configuration *again*. You're left to that if you don't know how the system works.

[reply] [top]

---

[»] Re: Make it easy for lazy people? Nahhh!
by Guillermo Sobalvarro - Oct 12th 2002 10:40:50

Stick to M$ products if they are not too hard for you. You seem to be an end-user so if these work for you don't delve into anything "hard". That way you won't have any commitment to anything. All you are interested in is getting your work done, right? Slam-bang.. et al. Most Linux and FreeBSD versions are easy to install and will give users something they can work with on a day to day basis from the start. If all you want are tools to get your job done they are there. I just don' see what is hard or difficult about it. Some people never get the hang of using a computer or an office suite. They should stick to pencil & paper and a calculator. Computers are not for everyone.

[reply] [top]

---

[»] Re: Make it easy for lazy people? Nahhh!
by djnietzsch - Oct 4th 2002 15:00:08

This is a really bizarre post. The poster seems to suggest that there is something noble about using unnecessarily difficult systems. Things should be exactly as difficult as they need to be and no more. For example, command line programs have a standardized (more or less) way of expressing parameters. You type the name of the command, some options, and some arguments. It's basically the same format for every program. And if you haven't memorized every option, you can just bring up a manpage that describes all of the options and details (again in a way that is fairly uniform from program to program). Some people would say this is not easy, and sometimes they're right. Some things get pretty cryptic. But it is straightforward. There's not much guesswork about format or what to expect in general. Configuration files seem to be a different matter. There are as many formats as services and there aren't a lot of universally applicable principles. This means that if you want to do some nontrivial configuration of something you've never configured before, you need to learn the format and syntax of the configuration files in question without much of a starting place. If there were a universal format/system for creating and editing these files you wouldn't have to start from scratch and things would be straightforward. That doesn't make them easy. Some matters of configuration will still require some care, subtlety, and knowledge. I would propose that one should only have to know what program does to use (use in a broad sense that includes configuration and whatever else might go into the program) it. Knowledge of how to use it should follow easily from universal principles that apply to all programs within the domain we're considering. Your relationship with your computer (yikes) should be a good one. To carry the analogy a step further, your computer should not be emotionally needy or enigmatic. It should be straightforward to deal with. You should not feel like you are a good person for putting up with it.

[reply] [top]

---

[»] Re: Make it easy for lazy people? Nahhh!
by Kurt Olsen - Nov 5th 2002 19:51:04


> Use FreeBSD as an example and take it
> from there. This is for people who know
> the insides of their systems and not for
> superficial types who just wan't a
> slam-bang-thank-you-ma'm OS. FreeBSD
> requires a "relationship", a
> commitment, and not just a one night
> stand. This is not for lazy asses.
> The market wants people who know what
> they are doing, not people who know how
> to use a GUI. If you want dancing
> clips, Einstein and doggies, stick to
> XP! Guess that's what all the envy is
> about.
I have to disagree, an operating system is a simple thing spawning a million options. As a longtime developer I must say that anything that makes the OS easy to install and configure is the way to go because the majority of folks want to run applications.


[reply] [top]

---

[»] Re: Make it easy for lazy people? Nahhh!
by AstroMan - Nov 22nd 2002 19:42:20


> Use FreeBSD as an example and take it
> from there. This is for people who know
> the insides of their systems and not for
> superficial types who just wan't a
> slam-bang-thank-you-ma'm OS. FreeBSD
> requires a "relationship", a
> commitment, and not just a one night
> stand. This is not for lazy asses.
> The market wants people who know what
> they are doing, not people who know how
> to use a GUI. If you want dancing
> clips, Einstein and doggies, stick to
> XP! Guess that's what all the envy is
> about.
This is just the wrong attitude. I am a developer and
most of my day is spent working or thinking about
computers. However, most of the world does not
revolve around computers. We are the
minority and they are the majority. A PC is meant to be
a tool. And tools are meant to perform certain tasks.
Let's look at an example of a mechanic. If a mechanic
had two tools on his table and one was great but
needed to have 32 switches turned on while pulling 5
levers while the other tool was ready to go, which tool
would he choose? Even if the one that was ready to go
was less powerful and broke half the time, it still allows
the mechanic to get his job done faster. This is what
people want! Linux has come very far. I want to see it
go all the way. There is no reason that Linux cannot
take the desktop market with some more work.

The Linux community needs to go for the ease of use
features now. Until that happens, the average Joe Blow
will be happy to use the closed source propriety garbage
feed to them. Why, because it allows them to do what
they want without needing to become a geek like you or
me. Since Linux has come so far in such a short time, I
think it is now time to finish it.

Let Linux reign.

[reply] [top]

---

[»] Webmin
by HandyCannon - Sep 28th 2002 19:22:52

http://www.webmin.com

Er, what's wrong with webmin? It's gui in the sense
it works in a browser, which also makes the front
end cross platform. It isn't just tied to Linux either,
and works across all Linux distros afaik.

Why re-invent the wheel....

[reply] [top]

[»] Re: Webmin
by Justin Yackoski - Sep 29th 2002 22:19:28

There are some tools that are good at a lot of things. Webmin is one of them. Webmin has some flaws which detract from its usefulness to some extent. First, it is only web-based, which although that is probably the best interface to pick if you're going to focus on a single one, picking only one can also be bad.

Second, webmin requires a module be written for each new software, and that this module be added by the user. Our system will (ideally) handle this automatically as version-appropriate files will be included with each software package.

Third, webmin doesn't make it terribly easy to allow people to write new modules. There are some things built in, but not nearly enough. There's little reason why someone should have to write more than a basic text file to explain how a piece of software (or part of the system) is configured. We want our system to be easy and straightforward to use for everyone involved, without keeping people too far from the details. For example, we may want to display someplace in most of our interfaces a note explaining what actual config file the user is editing should they want to edit it by hand.

Our system is specific to the task of configuration, but it will be general enough so that it won't need to be re-written to add support for new things. The process will be very simple, and may even be mostly automatic.

[reply] [top]

---

[»] Re: Webmin
by Martijn - Oct 29th 2002 07:14:02


> There are some tools that are good at a
> lot of things. Webmin is one of them.
> Webmin has some flaws which detract from
> its usefulness to some extent. First,
> it is only web-based, which although
> that is probably the best interface to
> pick if you're going to focus on a
> single one, picking only one can also be
> bad.
>
> Second, webmin requires a module be
> written for each new software, and that
> this module be added by the user. Our
> system will (ideally) handle this
> automatically as version-appropriate
> files will be included with each
> software package.
>
> Third, webmin doesn't make it terribly
> easy to allow people to write new
> modules. There are some things built
> in, but not nearly enough. There's
> little reason why someone should have to
> write more than a basic text file to
> explain how a piece of software (or part
> of the system) is configured. We want
> our system to be easy and
> straightforward to use for everyone
> involved, without keeping people too far
> from the details. For example, we may
> want to display someplace in most of our
> interfaces a note explaining what actual
> config file the user is editing should
> they want to edit it by hand.
>
> Our system is specific to the task of
> configuration, but it will be general
> enough so that it won't need to be
> re-written to add support for new
> things. The process will be very
> simple, and may even be mostly
> automatic.
Hi,

I agree with what you say, mostly, but what I don't think there is anything within the webmin architecture that precludes you from implementing the stuff you miss.

As I understand it, the webmin engine does most of what you need, and it would make a lot of sense for your considerable talents to be joined with the webmin krewe, rather then starting (yet another) configuration project.

Another thing I missed is how this system can be used to manage large, distributed networks, and what facilities there are to make this work.

[reply] [top]

---

[»] Sounds good
by Ritchie Young - Sep 28th 2002 18:17:06

Would you make the storage of the settings pluggable? Would you consider using LDAP as your store?

PCs and servers set up within a large (or small) organisation may be best served by keeping the configuration in an LDAP enabled directory (eg OpenLDAP). It's a standards based replicable directory. The administrator doesn't have to go to a machine or even connect to a machine to configure it.

It may also make sense to incorporate the possiblity of configuring other devices. Eg Unix servers, Windows servers and workstations, printers, etc) from the same console/store.

[reply] [top]

[»] Re: Sounds good
by Justin Yackoski - Sep 29th 2002 21:53:30

LDAP should not be the actual store for the data. The config data will remain in its native format to allow hand-editing, backward compatibility, etc. We did think about using LDAP as one of the interfaces, which could work well since it is designed to be able to edit such data. The LDAP server would merely read auto-generated XML made by our system and pretend to be actually doing things, when in reality it is just sending the modified XML back to our system. LDAP would be a good choice to offer, although due to the additional requirements (an LDAP server and knowledge of LDAP), it is currently not one of the primary interfaces and so will be developed later (or if you want to help, go for it!)

[reply] [top]

---

[»] Toybox
by Ariejan de Vroom - Sep 28th 2002 17:30:25

That's why, in my opinion, Linux is just a toy-O/S. Everyone builds his own Linux, and doesn't pay any attention to compatibility with other distributions, or any other standard there is.

I'm a fanatic FreeBSD user, and the best thing about FreeBSD is its purity. No code bloat, just a very fast and stable kernel, and everything is in places where it's logical to store them.

Maybe it's too late for Linux distro's to come clean, and produce some sane standards? I'll just keep to FreeBSD.

[reply] [top]

[»] Don't save state!!
by dentar - Sep 28th 2002 16:13:44

The problem with Linuxconf style configurators is that the configuration program attempts to "save state" elsewhere other than the configuration files themselves. This causes problems in a multi-administrator environment or in an instance where the admin wants to edit the file by hand. Next time the admin wants to go into Linuxconf (for example) to fix something not related, the Linuxconf program tries to sync up everything, thus throwing off what the administrator did with the file by hand.

[reply] [top]

[»] Re: Don't save state!!
by Justin Yackoski - Sep 29th 2002 22:11:30

I should point out that Linuxconf says on its website (sorry, I can't find where at the moment) that for some time (a few years I believe) it hasn't done that. Pretty much everyone agrees that this is a bad idea. All of the XML files used by our system are automatically generated as needed and the native config files are always treated as the authoritative copy.

Last I knew linuxconf did try to restart various services pretty much every time I used it for un-related things. I think that is a pain, and we'll be sure to avoid that except when necessary. For example, if someone changes a samba config directive, our system may restart samba automatically, remind the user to restart it, ask the user whether or not to restart it, or just shut up and assume the user knows that (this could depend on the user's preferences).

[reply] [top]

---

[»] Config file as File System
by DataMoist - Sep 28th 2002 15:11:00

Turning configuration files into a whole filesystem could be an end in itself. Consider:

- Hans Reiser wants to make the filesystem intelligent so the an XML document could look like a directy of files. He will put the XML-to-directory-and-back translation into the filesystem. No code for you to write. (Poor Hans, doing our work. ;)

- Logging and archiving configurations would be a job for tar (or CVS). No new code to write. (ok, maybe a cron job and a nice 'restore configureation' GUI that calls untar.)

- Access control (ACLs) already exist, and will eventually become standard (just like journaling file systems have). Neither the application nor the C4G program will need any special permissions-checking code -- the filesystem will do it. GUIs may be needed to help the user delegate tasks, but the hardend sysadmin will just use chmod or chacl.

- Task automation (such as "delete the /tmp share from samba") don't need parse code anymore. These tasks can now be done with simple shell scripts.

- Remote administration would be a simple network mount. No new protocols, no new code. Accessing and changing data when the computer is down is alreay supported by file systems like coda and intermezzo. Until those become more robust and standardized, a file-sync program like unison can be used. (The advantage over normal config files: fine-grained sync reduces the conflicts).

--
-- DataMoist -- The revolution will be packetized

[reply] [top]

[»]