How to Fix the Unix Configuration Nightmare
 by Matthew Arnison, in Editorials - Sat, Feb 16th 2002 00:00 PDT

Unix is steadily evolving into something much easier to use. The trick is to find tools that make things friendlier, but which fit in well with existing tools and are easier for people to take and use for new projects.

The Linux distributions and the BSD ports system are like this, taking an existing wealth of software and working with its quirks to streamline the installation and upgrading of applications and system software.

The GNU project has done a lot of good work in standardizing the compilation of software on different operating systems (automake and configure), parsing commandline options (getopts), and working with internationalization (gettext).

One thing that's still a mess, however, is application configuration.

There are useful general trends, like putting system-wide configurations in /etc and personal preferences in files beginning with . in your home directory. However, there is a complete zoo of configuration file formats. Some use the Windows .INI format. Others use simple variable assignments. Some use a natural language syntax. A certain amount of diversity is healthy, but in this case, it makes it a bit of a black art to build user-friendly and powerful configuration tools for all the software on a machine.

There have been various noble attempts to fix this.

Some popular packages, like Apache and Samba, have had point-and-click frontends built for them, but each frontend works differently. Other packages (notably the Linux kernel and Debian's debconf) have a range of interfaces that can be used, depending on the user's taste, from a raw text file and vi, to a text-based menu system, to graphical point-and-click. Another approach is to custom build a consistent set of frontends for a lot of different software configurations, as is done with the GNOME and KDE control panels, the Linuxconf project, and The Dotfile Generator.

I don't think I need to go into much detail as to what a mess the Windows registry is. It does provide a certain amount of power for graphical applications, but the text-level access is shoddy and cryptic, and a binary database seems to be an inherently unstable and inflexible way of storing data.

A new approach

All of the above efforts have made things better, but none have quite pulled off the kind of user interface consistency that Windows and MacOS have. However, despite an apparently messy situation, I think that Unix may not be very far from a workable solution.

Obviously, it's possible; MacOS X has pulled off a very nice graphical frontend to Unix. However, that frontend is the bit they're keeping closed, so the other Unix systems are going to need to build something Free.

The solution, I think, lies in combining the best of the approaches I listed above. It's vital that it be something that can work with existing programs, but which is attractive to developers to incorporate into new versions and new projects. Without integration into applications, it's always going to be a bit of a dodgy hack (see Linuxconf) that makes it difficult to change the configuration file in a variety of ways. Getting the developer involved is crucial for making sure the configuration manager keeps up-to-date with changes in the application, which I believe is a big part of the success of standards like gettext and configure.

It needs to accommodate diversity (the opposite of the One Registry approach, which I think will never fly on Unix), and it needs to be tenacious; it should be useful even with minimal information, but take good advantage of as much information as is given. It needs to help both the user and the developer. It helps the developer by automating and cleaning up configuration handling and documentation. It helps the user by providing flexibility and consistency.

Here's an outline of how it might work:

• A core system would handle parsing, verification, and storage of text-based configuration files in one or two basic formats.
• The master copy of the configuration is always left in the native text files (in /etc and ~/.*). This is where Linuxconf fell down; it started keeping its own copy of the configuration, which meant that if Linuxconf took over your system and then something stuffed up, it was difficult to edit a text file without losing Linuxconf completely.
• In order for some features to work, it might be necessary for application developers to switch to the use of the configuration manager for their internal routines. However, this should be possible without breaking backward compatibility with the original configuration file format for most features.
• A key element would be the configuration format description file. This would list the configuration options for a given piece of software, giving for each one the name, type (boolean, list, string, etc.), options, category (for subsections within the config), and help text (short and long).

  It would be good to be able to write descriptions of how options relate to each other. For example, one option might disable a bunch of other options.

  Ideally, you would be able to add optional hints to help with the layout of the options on the screen under various configuration editor systems.

  Done well, this could be a big attraction for developers, because it could automate a lot of the work involved in managing configuration systems. By creating a single place for storing such info, it could be used to automatically generate and update all documentation: user guides, man pages, Web page help, commented template systems and user config files, and programmatic help output.

• Because so much language is involved in configuration options, it needs to be internationalized from the ground up.
• This core system would be kept small and efficient, so that it is easy to add it to all sorts of projects without weighing them down. It would have interfaces in all of the common Unix languages: C, C++, Perl, PHP, shell, Python, etc.
• Plugins would allow the addition of a lot of diversity, and would be of at least two types:
  • File format plugins would allow the programming of new configuration file formats. This includes parsing, verification, editing, and writing. You could expect heaps of such plugins, preferably provided with an application or by a third party.
  • Editor plugins would give various methods for changing configuration files. These could range from a wrapper on your favorite text editor, to text-based menus (curses), to graphical menus (GNOME, KDE, etc.), to distribution-specific methods (e.g., Debian's debconf), to Web forms (useful for Web applications that have a lot of configuration options). It should be possible to invoke an editor from within an application (using the application's File -> Preferences... menu, or a commandline option) or directly from the commandline (or a GUI equivalent that lists installed apps, like a control panel). It should also be possible to programmatically edit configurations, either through a standard commandline or through the various language bindings listed above. Network administrators would love such tools.
• It all needs to be language, distribution, and operating system neutral, so as to avoid turning off any potential software developers who might find it useful.

With such a system, Unix could go even further than Windows and MacOS, providing the ease-of-use of graphical configuration with the flexibility and security of remote text-based configuration and the power of programmatic access to read and change configurations throughout the system.

If this seems like a good idea, all we need is a team of programmers and documenters to write it. It seems like a lot of work, but with the right touch, and strategic initial energy, it would be something that would attract contributions from a lot of different angles.

Author's bio:

Matthew Arnison is passionate about science, activism, and media. He has played a major role in the setup and running of the activist media projects cat.org.au, active.org.au, and indymedia.org, all of which rely on Free Software. He first installed GNU/Linux in 1995, and since then has helped set up and maintain several Internet servers, including plenty of seat-of-the-pants action during live indymedia Webcasts. When programming, he tends to concentrate on user design, and he likes to think an emphasis on usability in the active software is part of the success of indymedia.

T-Shirts and Fame!

We're eager to find people interested in writing articles on software-related topics. We're flexible on length, style, and topic, so long as you know what you're talking about and back up your opinions with facts. Anyone who writes an article gets a t-shirt from ThinkGeek in addition to 15 minutes of fame. If you think you'd like to try your hand at it, let jeff.covey@freshmeat.net know what you'd like to write about.

[Comments are disabled]

Referenced categories Topic :: System :: Installation/Setup Topic :: System :: Systems Administration

Referenced projects Debian GNU/Linux - The Universal Operating System. GConf - A configuration storage library. Linuxconf - A sophisticated administrative tool. Webmin - A Web-based interface for Unix system administration.

 Comments

[»] COAS
by sharkey - Mar 25th 2002 23:20:52

This sounds a lot like the COAS project started at Caldera. <www.coas.org> Like all things Caldera, it failed for lack of effort, but perhaps someone could pick up the pieces and make it work.

[reply] [top]

[»] why there is a problem?
by karim kodera - Mar 19th 2002 16:19:09

Well i have read most of the comments posted about the unix configuration nightmare along with many comparisons with the windows system. i guess the problem is devided in 3 parts.

1) system level configuration ( fstab , inittab, termcap ,..................);
2) system level service configuration ( bandwidth management ,firewalling , printing, ......)
3) programs configuration


1) for system configuration files i guess all we need just write a standard api interface such as

addIdeDrive( .............)
setAutoMount(...........)

things like that and every distro must have this api set . Normal program such as adduser, passwd and such basic programs should also use this api's. Also there must be a certain file permission that allows only kernel level process to modify these files in order for them not to be modified by hand. I guess this would not take too much time because basic system configuration files are not that numerous.

2) as for the system level services configuration problem. i was and still am writting a program for intefaacing the tc command line. the documents about tc are pieces of shit. some of them tells you what is completly the opposite of the others. so develeper of such services ( specially new ones ) should create more friendly interfaces along with a programming api interface too. so that things can be easily interfaced later

3)as for the programs configuration problem .it lies in the open source projects. Almost all developers of the open source projects do not give much attention to the configuration of their programs. Take blackbox as an example it is a wonderfull window manager configuration is a nightmare. another one is window maker too much configuration before you can really run it, now kde just runs as soon as you start it. so who is the most popular kde. that is why windows is popular not because the system is popular but because application are much easier to configure if my configuration is hard my software would not sell. But for linux we use a lot programs only because there is nothing else to
use to acconplish a specific task. So the main idea is when a software developer creates a new program he should make a configuration program along with it. i guess that will solve many of our problems.

In the end, I want to say computer is about programs. Programs should do the stuff .If i have a program to configure things i don't really have to care how the configuration is written, where it is written or whether it is centralised or distributed. actually if i did not know that windows has a registry I would have never guessed. I never needed direct access to it as a user.

[reply] [top]

[»] Re: why there is a problem?
by Travis Gerspacher - Mar 21st 2002 03:04:42

3)as for the programs configuration
> problem .it lies in the open source
> projects. Almost all developers of the
> open source projects do not give much
> attention to the configuration of their
> programs. Take blackbox as an example it
> is a wonderfull window manager
> configuration is a nightmare. another
> one is window maker too much
> configuration before you can really run
> it, now kde just runs as soon as you
> start it. so who is the most popular
> kde. What is so hard about configuring blackbox?
You shouldn't need a great big GUI in order to figure out how to configure something like this. Most people out there can look at the below line and figure out what it would mean.

[exec] (Galeon) {/usr/bin/galeon}

This is my biggest gripe against a lot of the current Linux development. GUI developement has been psychotic lately. I keep seeing bigger and bigger GUI's that are only serving the purpose of making easy things harder.

I have used KDE before and it was was real pretty and a huge fucking waste of space. In order to change the menu you have to root through a huge list of menu items that link to things you will never use just to edit the menu. KDE does not turn on as soon as you start it, you have to wait for it to boot because it is large enough and complicated enough to be it's own operating system.

I am can think of another thing in KDE that follows this same patterm too. Kpackage - gimme a break. If this application could run long enough without crashing, then I might be able to run a really overdone program just to install a simple rpm. c'mon, what is so hard about opening up your trusty xterm and typing 'rpm -i <packagename>'.

Developers created a useless program to keep someone from having to type in 5 characters and the name of the package.

I am guessing that KDE is the most popular for the same reason windows is so popular. People don't know any better.

[reply] [top]

---

[»] Re: why there is a problem?
by GuodMan - Jun 13th 2002 01:22:25

RPM files are a much bigger problem than config files, and the advantage of the good package managers (like rpmdrake) is that they try to automatically install or remove packages to satisfy dependencies (they do fail often). The idea of going to a command line and opening a file scares many people. If there is a nice gui, the users can make modifications and be sure that they will not mess up the program by typing something wrong. before you start talking about how easy it is (and I would agree that it can be easier) remember that when you go to the general public, many people don't want to spend any time learning about their computer. With a gui, all of the available options can be listed for the user to pick from, and the user doesn't need to search for the file to configure.

[reply] [top]

---

[»] It's a good idea... BUT
by BlueEAGLE - Feb 22nd 2002 14:48:25

I believe that any piece of software that requires configuration should have a configuration feature buildt in. This is fine for any new software you find, but what puts many new *nix users off is the fact that they don't really know where to start.

Even the most trivial thing as running fortune on login had me put off in the beginning. (i looked for something that looked like autoexec.bat... but don't tell anyone). I found out how after spending n hour reading x^n lines of howtos and docs. After a while I couldn't remember what I was looking for...

This is where imho someone like the distor vendors ought to come into play. Or something as easy as an index to what config files does what.

Here info has started a good thing when it comes to indexin and making an overview of man pages.

As to the format of config files... Comments in the beginning ought to explain them... Well config files shouldn't have to be tampered with by hand at all!

I hope this makes any sense... Forgive me if I'm rambeling but it has been a long day:)

--
This is provided "As-is" without any waranty, not even the implied waranty of making any sense what so ever *s* Yours Terje "BlueEAGLE" Monsen

[reply] [top]

[»] unixconfig.sourceforge.net and mailing list now open
by Matthew Arnison - Feb 22nd 2002 06:30:23

some of the comments are very interesting. i really appreciate all the ideas flying around here.

we've created an ongoing web site and mailing list for people to continue discussing, organising, and hopefully in the end coding. it's at unixconfig.sourceforge.net.

meanwhile here's another link to the permanent home for this article. this is where i'm making revisions and adding links.

[reply] [top]

[»] Re: unixconfig.sourceforge.net and mailing list now open
by Dwight Walker - Jan 3rd 2007 23:13:54


> some of the comments are very

> interesting. i really appreciate all the

> ideas flying around here.

>

>

>

> we've created an ongoing web site and

> mailing list for people to continue

> discussing, organising, and hopefully in

> the end coding. it's at

> unixconfig.sourceforge.net.

>

>

>

> meanwhile here's another link to the

> permanent home for this article. this is

> where i'm making revisions and adding

> links.

>

That site link is dead. I tried search sourceforge.net for unixconfig and got nothing.

--
WWWalker Web Development Pty Ltd

[reply] [top]

---

[»] Daemon.
by Olivier Rossel - Feb 22nd 2002 04:53:26

I realmly like the webmin system.
I wonder if a kind of system could be done that would be:
a daemon listens on a specific port (a port for configuration) on localhost.
An abstract layer handles a /etc/whateverfile.conf to "standard XML" translation (both sides).
The daemon handles the serving of thoses standard representation of configuration datas. And clients are available to format that XML into GUI, and command the daemon to write the modification into the corect file.

The idea is to separate tasks as in a client/server system.

[reply] [top]

[»] How about WBEM?
by Viktor Mihajlovski - Feb 20th 2002 10:37:37

Sorry for the long posting, but it's really a matter of heart to me...
The problem with configuring (not only) applications has multiple dimensions. One of them - as already pointed out in different posts - is access technology. A single, consistent method is necessary to read and update all the configuration data. I chose the term "access technology" deliberately because we cannot not really expect all the legacy /etc files like inittab and passwd to go away or change their format (be it XML or something entirely different). We also cannot faithfully expect that formats converge, meaning one is given up in favor of the other. This means that the access technology must facilitate a wrappering approach to the current configuration repositories. It also has to be suited for scripting, fancy GUIs of course and allow remote access, especially important if we think of bigger installations.
Coming up with such a technology is conceivable, it will buy us only part of the solution however. The problem is that even if all config data was accessible in a single way, we are still missing semantical information (ever tried to configure a printer via, say the Windows registry?). What this really means is that a data model is needed in addition, describing the semantics of configuration data (and other parts of the system). The data model must provide different levels of detail, giving the possibility to perform high-level, generic configuration tasks as well as highly application-specific modifications.
Let's take web browsers as an example: all of them allow the specification of proxy servers, each in it's unique way. Now assume a situation were a company-wide proxy server changes and hundreds of clients have to be reconfigured to reflect the change. In this case it's really convenient perform a remote "set proxy for client xyz" configuration command, without even knowing the exact nature of the client's browser.
Access and data modeling technologies for systems management purposes exist today already, in form of DMTF's WBEM and CIM standards. Parts of the data models therein deal with configurations and settings (of devices and applications) which I believe to be a sound base for further developments. Now what does this mean? For the purpose of application configuration, appropriate data models have to defined by extending the respective CIM models. Then it is necessary to write data access modules interfacing to the real resources, the so called providers. And what's perhaps most important, management applications exploiting the stuff.
Several Open Source projects related WBEM technology are cooperating under the umbrella of the WBEMsource initiative. If you want to learn more about, http://www.opengroup.org/wbemsource would be a good starting point, as well as the DMTF website http://www.dmtf.org . To summarize and emphasize my statement: the infrastructure technology is there, the hard work still has to be done. It will take quite some time, but if successful it will lead a way out of all the configuration quirks and nightmares, and this not only for Linux but for all platforms supporting WBEM.

Regards, Viktor

[reply] [top]

[»] Be practical
by Jonas Bofjall - Feb 20th 2002 04:13:52

It's not very clear what you want. Any GUI-related options under X11 are set in the (centralized, which "won't fly"?) X Resource DB. Do you wish to have each app's configuration separated, or each type of configuration separate (as in xrdb)?

Being able to set default configuration for lots of apps is a good thing (IMHO). The problem as I see it today are machine-generated configs which are not human readable. (One more XML-config and I'll puke! It only requires this-n-that many libs to parse...)

So please start a GNU project for cross platform, human readable configuration files which supports site defaults, system defaults, getopt overrides and inter-app defaults (such as colors, mailbox location, printer types etc.), and I'll use it any day!

(As you understand, I believe this is a large undertaking so you'll need the weight of GNU and KDE behind you...)

[reply] [top]

[»] avoid a global registry for local apps
by SillyConMan - Feb 19th 2002 22:39:22

One of the biggest problems with having a single global registry like in Windows is that if you do a clean re-install of the OS, or a from-scratch install of an upgrade, you lose everything, including applications you may have downloaded and installed separately.

One of the nice things in Unix is that you can have /usr/local on a separate partition/filesystem, and that way when you do a clean re-install of the OS, you can avoid formatting /usr/local, and so continue using all your existing applications in that directory. This is possible because the configuration information for the applications was stored on the same filesystem as the application itself. If some important configuration information were to be stored in a "global" place (e.g., in /etc), then this advantage would be lost.

[reply] [top]

[»] Re: avoid a global registry for local apps
by Phil Jones - Feb 20th 2002 06:47:42

That advantage has a high price, namely, extra work to do a full system-configuration backup and the possibility of missing something. In this discussion there are people saying they switched to Debian because it puts things nice and organised in /etc. You can put /etc on a separate partition if you don't want to format that, I guess.

[reply] [top]

---

[»] Re: avoid a global registry for local apps
by Travis Gerspacher - Feb 21st 2002 19:03:38


> That advantage has a high price, namely,
> extra work to do a full
> system-configuration backup and the
> possibility of missing something. In
> this discussion there are people saying
> they switched to Debian because it puts
> things nice and organised in /etc.
>
> You can put /etc on a separate
> partition if you don't want to format
> that, I guess.
I have always thought putting /etc on a seperate partition would be handy but if I am not mistaken /etc cannot be on it's own partition. I think the same goes for /lib and /dev. Please correct if I am wrong.

I personally hate having a programs config file in /usr/local.

It has always been my opinion that config files go in /etc, logs go in /var/log and binaries go in a reasonable location. I try to adhere to the FHS in placing binaries but that doesn't always work out.

I understand your point about how nice it is to have config files right next to the binaries but I like to have everything in /etc to make it easier to find stuff. I think it is nice to go into /etc and be able to configure everything. The /etc directory is usually pretty small so it is trivial to backup. (I just did a tar -cvzf on my etc directory and I ended up with a 595K tarball.)

For me a reinstall ends up doubling as an upgrade and I run into trouble using old binaries with new libs, so what I do is do a full upgrade and get everything installed and then I piece the backed up config files back into place. Config files are almost always portable between upgrades or will only need tweaked a little bit.

I usually change certain packages around so that things end up in the right place. Programs that come to mind are apache and qmail, which like to put all of their file under one subdirectory.

This way instead of trying to remember where you installed something, you only need to think about the type of file. (If it is a config file then it is in /etc, dynamic information would be in /var, etc, etc, etc.)

Another easy way of soing things is putting /usr/src/ or /usr/local/src on a seperate partition and keeping ALL of your source on it. so after the reinstall all you have to do is a make install and things are ready to go. I keep a copy of /etc on this partition too. This helped me do a complete reinstall and have all available services back up in about 2.5 hours on my home server/workstation.

[reply] [top]

---

[»] Re: avoid a global registry for local apps
by John Stalker - Feb 27th 2002 12:23:20


> I have always thought putting /etc on
> a seperate partition would be handy but
> if I am not mistaken /etc cannot be on
> it's own partition. I think the same
> goes for /lib and /dev. Please correct
> if I am wrong.
You are correct. The reason is very simple. If
/etc is in a separate partition it will never get
mounted, because you don't have an /etc/fstab to
tell mount where it is. There are some hacks you
could use to get around this, but all of the ones I
can think of are pretty unpleasant.



[reply] [top]

---

[»] Re: avoid a global registry for local apps
by KJK::Hyperion - Jan 26th 2003 09:02:31


> One of the biggest problems with having
> a single global registry like in Windows

no, it isn't a single, global registry. It's split up into many system-wide parts - namely, SOFTWARE, SYSTEM, SAM, and SECURITY - and a per-user part. Notably, the per-user part is stored in a file located in the user's profile directory. Not to lose your installed programs, you backup SOFTWARE. Not to lose drivers and services, you backup SYSTEM

And it can be done better than this

--
make a difference - use Windows! (http://www.reactos.com/)

[reply] [top]

---

[»] I want a standard
by Phil Jones - Feb 19th 2002 07:12:41

I want a standard which other programs can use to:

1) find out what options my program has
2) set those options.

I do wish there was such a standard. I have noticed that other programs which depend on mine tend to say "Have you enabled X in createusers.conf?", "Have you enabled Y in createusers.conf?", etc.

If there was such a standard then those other programs would be able to check and enable it themselves. Those programs could query my program to see what options have been set, and perhaps change them as necessary.

Such a standard might allow my program's options to be automatically in a magical '/proc/virtual-etc' preferences virtual filesystem - without the need for any additional programming. (This virtual filesystem is described in my post below).

I don't know what form such a standard might take, but if it delivers this dream, I will be quick to adopt it in my program.

[reply] [top]

[»] Re: I want a standard
by Izak Burger - Feb 21st 2002 04:20:54

All I can say is: Yech. Keep application dependent stuff like this out of the kernel please. Also, Please, NO CENTRAL REPOSITORY! Then we have a single point of failure exactly like the windows registry.

Rather use something like gtk-config. Then your application can simply call otherapp-config --show or something like that to figure things out.


> I want a standard which other programs
> can use to:
>
> 1) find out what options my program
> has
> 2) set those options.
>
> I do wish there was such a standard. I
> have noticed that other programs which
> depend on mine tend to say "Have
> you enabled X in
> createusers.conf?", "Have you
> enabled Y in createusers.conf?",
> etc.
>
> If there was such a standard then
> those other programs would be able to
> check and enable it themselves. Those
> programs could query my program to see
> what options have been set, and perhaps
> change them as necessary.
>
> Such a standard might allow my
> program's options to be automatically in
> a magical '/proc/virtual-etc'
> preferences virtual filesystem - without
> the need for any additional programming.
> (This virtual filesystem is described in
> my post below).
>
> I don't know what form such a standard
> might take, but if it delivers this
> dream, I will be quick to adopt it in my
> program.


[reply] [top]

---

[»] How about extending /proc?
by Phil Jones - Feb 19th 2002 06:20:08

The Linux kernel puts its configuration variables in the virtual filesystem /proc. I think this is a winning strategy. It fits in with the fundamental Unix design brilliance, 'everything is a file'.

I suggest building on the /proc filesystem. I suggest a virtualised filesystem as a consistent interface to configurations such as user accounts and Apache setup. The virtual filesystem would read and write the normal config files in the /etc directory.

Restoring a backup of /proc/virtual-etc would ideally have the same effect as restoring a backup of /etc. The only difference would be the former would take longer because the 'deep magic' running the /proc/virtual-etc filesystem would have to build the real configuration files in /etc.

The /proc filesystem is brilliant. I can browse the hierarchy of system configuration information using simple shell commands or the browser of my choice. I can see details such as the CPU info using any editor, or using a simple 'cat' command.
I can even turn on and off kernel facilities, such as IP forwarding, using a simple 'cat'.

So, extending the genius and simplicity of /proc, how about something like this?

Example 1: View all the Unix accounts in the system:

shell#> ls /proc/virtual-etc/unixaccts/users
root mail fred joe barney wilma

Example 2: Set the shell of the Unix account 'fred' to '/bin/bash':

shell#> cat "/bin/bash" > /proc/virtual-etc/unixaccts/users/fred/shell

Example 3: Copy a named virtual host in Apache, and set the HTML directory for the new virtual host:

shell#> cd /proc/virtual-etc/apache/vhosts
shell#> cp -R somehost newhost
shell#> cd newhost
shell#> cat "/var/www/newhost" > DocumentRoot

[reply] [top]

[»] Re: How about extending /proc?
by Jared H. Hudson - Feb 20th 2002 01:48:32

I really like this idea. In my mind this would allow applications to still keep their configuration files where/and how they want while ALSO creating a centralized place for all configuration information. I would propose that the best way to do this would be a linux kernel virtual filesystem in conjunction with a daemon that has multiple modules for different applications and their configuration file formats. That way people (or application programmers) can provide a configfs module for their application, if they don't someone who understands how to configure the application can write their own and have configfs dlopen() the module for that application.


> The Linux kernel puts its configuration
> variables in the virtual filesystem
> /proc. I think this is a winning
> strategy. It fits in with the
> fundamental Unix design brilliance,
> 'everything is a file'.
>
> I suggest building on the /proc
> filesystem. I suggest a virtualised
> filesystem as a consistent interface to
> configurations such as user accounts and
> Apache setup. The virtual filesystem
> would read and write the normal config
> files in the /etc directory.
>
> Restoring a backup of
> /proc/virtual-etc would ideally have the
> same effect as restoring a backup of
> /etc. The only difference would be the
> former would take longer because the
> 'deep magic' running the
> /proc/virtual-etc filesystem would have
> to build the real configuration files in
> /etc.
>
> The /proc filesystem is brilliant. I
> can browse the hierarchy of system
> configuration information using simple
> shell commands or the browser of my
> choice. I can see details such as the
> CPU info using any editor, or using a
> simple 'cat' command.
> I can even turn on and off kernel
> facilities, such as IP forwarding, using
> a simple 'cat'.
>
> So, extending the genius and
> simplicity of /proc, how about something
> like this?
>
> Example 1: View all the Unix accounts
> in the system:
>
> shell#> ls
> /proc/virtual-etc/unixaccts/users
> root mail fred joe
> barney wilma
>
> Example 2: Set the shell of the Unix
> account 'fred' to '/bin/bash':
>
> shell#> cat "/bin/bash"
> >
> /proc/virtual-etc/unixaccts/users/fred/shell
>
> Example 3: Copy a named virtual host
> in Apache, and set the HTML directory
> for the new virtual host:
>
> shell#> cd
> /proc/virtual-etc/apache/vhosts
> shell#> cp -R somehost newhost
> shell#> cd newhost
> shell#> cat
> "/var/www/newhost" >
> DocumentRoot
>


[reply] [top]

---

[»] Re: How about extending /proc?
by Phil Jones - Feb 20th 2002 06:30:10

I could make a little proof-of-concept application. It would be a simple program. It would make a RAM disk and mount it someplace. Then it would go through /etc/passwd (for example) and create a file structure like this:

virtual-etc/
virtual-etc/unix-accts/
virtual-etc/unix-accts/users/fred/
virtual-etc/unix-accts/users/fred/primary-group
virtual-etc/unix-accts/users/fred/realname
virtual-etc/unix-accts/users/fred/secondary-group
virtual-etc/unix-accts/users/fred/shell
virtual-etc/unix-accts/users/fred/uid


I could then watch /etc/passwd for changes and arrange for the RAM disk tree to be rebuilt when necessary. Secondly, any changes to the RAM disk files, maybe including comments, could be parsed and written back into /etc/passwd. I could set UNIX permissions on the files and directories in the RAM disk to control which users can read or write what.

The tree could be extended to cover /etc/shadow and /etc/group, if needed. Should I give this a go?

I hope a tiny partial implementation would be useful. For example, it might allow an application author to easily pick up selected Unix account info about Joe Random User without having to worry about the format or location of the /etc/passwd file.

[reply] [top]

---

[»] Re: How about extending /proc?
by Berge Schwebs Bjørlo - Feb 21st 2002 00:43:44


> I could make a little proof-of-concept
> application. It would be a simple
> program. It would make a RAM disk and
> mount it someplace. Then it would go
> through /etc/passwd (for example) and
> create a file structure like this:
If you could have done that, it would have been excellent. I love the idea of your comment's parent, and really think it's the way to go. A proof-of-concept application (or kernel module, I imagine) would, among other things, be cool to play with (-8 This is just what Linux/Unix needs to get the little push over the edge to world domination.. (-;

--
Berge S. Bjørlo Trivini Productions www.trivini.com - "Linux is not the one true solution... but we are working on it." -Alan Cox :wq

[reply] [top]

---

[»] Re: How about extending /proc?
by Fred - Feb 21st 2002 03:09:11


I think polluting /proc with a registry is bad, maybe /reg or /registry

Still, this doesn't solve the problem completely they are just files or pseudo-files.

Are you telling that values should be store like this?

/registry/Software/Company/VIM/params/options=hex:48,00,04,00,00,00,27,0c

Parsed by:
$entry =~ m/(\S+)=hex\s*:\s*([0-9A-Fa-f]+,?\s*)+/;
$key = $1;
$val = $2;

to mean this:

/registry/Software/Company/VIM/params/
options = $48,00,04,00,00,00,27,0c


Another way would be like Windows INI files or Windows REG files like this:

REGEDIT4 example
[HKEY_LOCAL_MACHINE\Software\Microsoft\Active Setup\Installed Components\{28FD0F82-4A73-4453-84A6-2F4F62702A3F}]
@="Windows Setup - XYZ"
"ComponentID"="PROGDL"
"IsInstalled"=hex:01,00,00,00
"Locale"="EN"
"Version"="5,4,1103,3"

I have only one problem with /reg or /proc what about NON-ROOT users ?

This is why I loved the SHTML/XML include:
<!--#include file="~/user_registry.xml"-->

Let say I want to install VIM on /home/fred/
because the sysadmin loves EMACS and hates VI, let say.
Will this means that the sysadmin can prohibit installation of software on /tmp or /home ?

Sincerely,

Fred.



[reply] [top]

---

[»] Re: How about extending /proc?
by Phil Jones - Feb 21st 2002 07:22:47

You wouldn't have to mount the 'virtual-etc' RAM disk directory structure in /proc. You could mount it anywhere you like.

The idea isn't a registry. It's just an alternative view of normal configuration files. Besides, I would prefer to keep the 'R' word far away. The 'R' word has way too many negative associations IMHO. :-)

For per-user prefs, I could maybe try mounting a small RAM disk somewhere in each user's home directory. For vi preferences, it might be possible to create a file structure like this:

/home/joe/.virtual-dot/vi/
/home/joe/.virtual-dot/vi/line-numbering
/home/joe/.virtual-dot/vi/shift-width
/home/joe/.virtual-dot/vi/wrap-margin


These files would dynamically represent various entries in the user's "~/.exrc" vi preferences file. If "~/.exrc" gets updated, the tree in the RAM disk would be updated automatically.

For example, you could see what has been set in "~/.exrc" using these commands:

shell> cd /home/joe/.virtual-dot/vi
shell> more *
*********
line-numbering
*********
# Controls whether line numbers are shown
0
*********
shift-width
*********
# Number of characters to use when indenting or outdenting
3
*********
wrap-margin
*********
# Place carriage return at the end of each line, starting this many characters from the right edge. If -1, no wrap margin is used, ie, a carriage return is inserted only when you type one.
-1


This says don't line number, use a shiftwidth of 3 characters, don't insert carriage returns automatically.

Line numbering could be turned on using this command:

shell> cat "1" > /home/joe/.virtual-dot/vi/line-numbering

This would update the "~/.exrc" file automatically. Should I give this a try?

[reply] [top]

---

[»] Re: How about extending /proc?
by Fred - Feb 21st 2002 23:09:21

Okay, so it's even easier than I tough the parameter are stored within the file of that name, where the line doesn't start with

# SOME COMMENTS

like shells for instance

Therefore, everything is done via cat... =)
cat "whatever" > /reg/key
cat /reg/key | grep ...


> shell> cat "1" >
> /home/joe/.virtual-dot/vi/line-numbering

You should give it a try, looks simple enough for anyone I think.

Still, there should be a commity for structuring
that tree, inspiring by MicroSoft registry
success and errors would be useful !

Another good point is that instead of having
crappy {12341-1243213-12432} links
we could use ln -s !

Give it a try man!
Cuz your da man!


Fred.

[reply] [top]

---

[»] Re: How about extending /proc?
by Phil Jones - Feb 22nd 2002 07:26:21

I made a ramdisk mounted on /vetc, plus a Perl script to read /etc/passwd and /etc/group and run the appropriate 'mkdir' and 'echo' commands. Then I made a few test users and put them in the 'users' group. The result was encouraging, even though I am not much of a Perl hacker:

shell> cd /vetc/unixaccts/users/phil
shell> ls
gid uid homedir realname shell
shell> more *
***gid ***
100
***uid ***
500
***homedir ***
/home/phil
***realname ***
Phil Jones
*** shell ***
/bin/bash
shell> cd /vetc/unixaccts/groups/users
shell> ls
gid members
shell> more *
*** gid ***
100
*** members ***
phil
user1
user2
user3


Maybe I should add parsing of /etc/shadow? If I did then it would be possible to do this:

shell> ls /vetc/unixaccts/users/phil
gid uid homedir realname shell min max warn
inactive expire

It took a few seconds to build the /vetc RAM disk directories and files but it did appear to be possibly useful despite being so limited. I was able to browse the results using mc.

Should I post the code here, or mail it to someone?

[reply] [top]

---

[»] Re: How about extending /proc?
by Berge Schwebs Bjørlo - Feb 25th 2002 17:44:10


> Should I post the code here, or mail
> it to someone?
If you could mail it to egrebNO@SPAMtrivini.com (remove the obvious), I would be greatful. It looks quite interesting to hack furher on, if my Perl-skills can do any good (wich they probarbly can't, since I'm still learing Perl). Keep up the good work! (-8

--
Berge S. Bjørlo Trivini Productions www.trivini.com - "Linux is not the one true solution... but we are working on it." -Alan Cox :wq

[reply] [top]

---

[»] Summary
by Fred - Feb 18th 2002 21:57:52

- *MUST BE* XML based
- XML to text converter
- text to XML converter
- XML editing is a joke if you use XML editors,
or some vi/emacs completion macro

- Registry should look like this:

/etc/registry.xml

<!-- This solves any global registry problems -->
<REGISTRY>
<!--#include file="/usr/local/myapp1/config.xml"-->
<!--#include file="/usr/local/myapp2/config.xml"-->
<!--#include file="/usr/local/myapp3/config.xml"-->
<!--#include file="/usr/local/myapp4/config.xml"-->
<!--#include file="~/user_registry.xml"-->
</REGISTRY>

- We need an easy method for shell script insert/query tool

#!/bin/csh
if ( -e /REGISTRY/app/myapp1/ ) then
setreg /REGISTRY/app/myapp1/key1 value
endif

We need a C API which is very simple to use:

char* value = registry_get( "/REGISTRY/app/myapp1/key1" );

registry_set( "/REGISTRY/app/myapp/key1", "value" );

struct registry_tree_t registry_select( "/REGISTRY/app/myapp/", "/^menu\w+/" );

register_insert( "/REGISTRY/app/myapp/key1" );

register_delete( "/REGISTRY/app/myapp/" );

In Perl, we could do the same, or simply
use a DBD::XML similar, i.e. DBD::AnyData

Other language should have something similar
to C or Perl

Another idea, is to use an SQL command-like
query engine.

If the C interface exist, ports to other language
should be easy.

How do you save the configuration for XYZ app ?

tar cvfz xyz_conf.tgz /usr/local/XYZ/config.xml

Fred

[reply] [top]

[»] Bridging the gap(s)
by Mark - Feb 18th 2002 17:31:34

It seems to me that there are a few steps that are needed to get to a unified configuration file format:

1. Decide on the file format
2. Write bridges to let existing, unmodified applications work with the file format,
3. Convert the existing applications to use a standard API that may also use the new file format.

These steps can be broken down:

1a. Decide on a basic file structure.
1b. Decide on a set of common elements.

I would suggest simply using XML Schema and XML files to represent the configuration. It doesn't mean that XML files need to be used to store the config, just that the config can be serialized to/encoded from standard XML without losing information.

2a. Define a schema for each specific application. If necessary, extend XML Schema to include helper fields.
2b. Write bridging tools to convert between the XML format and the native format used by the target application. In the case of XML, you could just write XSLT transforms. These are much easier to keep up-to-date than straight code.

Part 2 means that we are not locked into a schema immediately -- we can play around with the schemas until we are happy that they provide all the functionality of the original files.

3a. Write a simple mechanism for accessing the configurations. XML gives us XPath and the DOM, which means that writing configuration parsers is straightforward -- we have ways to refer to any given node in a configuration file and we have a low level API for accessing it easily. So really a configuration file parser is simply a wrapper around the DOM.
3b. Write a configuration editor for each tool. With XML this is simple, you just need a tool that can read the XML schema definition -- IBM's Xeena tool (though written in Java) provides context-sensitive pallete for editing an XML document based on the DTD, so doing the same for an XML Schema should be fairly straightforward.

The advantages of this scheme are:

1. All the basic tools already exist as free software.
2. Potentially 100% compatability with old applications
3. Most of the code is XSLT and therefore more easily maintained than a hard-wired parser (I am only arguing that XSLT is easier to follow than C code implementing a parser for the same structure).
4. Permits graphical or text based configuration
5. In conjunction with the kernel file monitor and a simple daemon, modifying the XML source of a configuration file could automatically cause the daemon itself to restart (obviously this would need to be configurable, too).
6. Very easy project to start; there are not thousands of lines of code to write.
7. 100% standards based, most of this stuff developers are already familiar with (or should be).

IMO the approach of linuxconf, Ximian Setup tools and Webmin is all wrong wrong wrong. Unifying the configuration can be done without initially rewriting the applications. Start from the bottom -- unify the configuration first and let the application authors decide when, if ever, to jump on board.

Finally, the file format could permit references to "included" configuration files, the storage of which could be in multiple places including over the network. For example, the standard configuration document (e.g. /etc/xcf/bind.xml) could include configuration from http://server.example.com/xcf/slave.xml. This would optionally permit configuration to be based on a single server or alternatively across multiple files.

[reply] [top]

[»] I like the idea
by Tim Mathews - Feb 18th 2002 17:31:30

I've been wanting to do something like this for a while. I've made a few comments above, but this one's my big rant.

This is something thats been needed since the eighties. If it had been done when the GNU project was formed, it would already have been a standard when Linux was first being put together.

There is a lot of archaic mess that Linux inherited from various other Unices, depending on which (if any) version particular developers were familiar with. So now GNU/Linux is a little of everything and not a lot of anything (standards wise).

I think that there are three big steps involved with making this happening.

First, review the specs for directory layout for AT&T Unix. Then review the way the BSD's do it, then compare layouts from the big 4 of Linux (Redhat, Mandrake, Slackware and Debian) and maybe SuSe as well.

Second, now that we have an idea of how different disto's do it, sit down and highlight the strengths and weaknesses of the various layouts. Then we can come up with a layout that will actually work well. This means two things, first the partition that has the config stuff on can be mounted read-only with out worry of messing anything up, and second it means that most user programs and help files can live on the network.

Thirdly, and I think this is the one point that everyone has been debating, develop a standard config file layout. I would strongly recommend not using XML and come up with a better way. There are several reasons for this. The first is that although XML is OK for use with a GUI, its no fun in a text editor. I don't think that an ncurses based UI is good here either. Ever try to use Linuxconf on a text only terminal over a packet radio link? Well written shell scripts can be just as easy, if not easier to read than XML, and its not too difficult to get a program to read them either.

A nice graphical config is good, and I think neccissary as well as an ncurses based one, but I think we still need the ability to easily tweak things from a text editor. With a standard API, a web based config would be a piece of cake. The webmin people would probably help out there.

I would be willing to coordinate a project like this, if anyone wants to help out, please contact me. I forsee the need for at least a half dozen people to start and then a lot more to get the config files set up the way we decide. Obviously, community input is essential so getting a website and mailing list going would be nice.

Being able to move from one version of Linux to another and maybe even on to BSD without having to think about how things are layed out would make a lot people a lot happier as well as make the transition to Linux a whole lot smoother (for windows people anyway). If you start to learn on one distro and then switch to another, you shouldn't have to start all over again. Being able to pick up where you left off would be a huge help.

I don't know if my email is on my profile, so if you want to contact me, its tim@accssden.dhs.NOSPAM.org, obviously, take out the ".NOSPAM" part.

--Tim

[reply] [top]

[»] LibCFG: A configuration management API
by Colin Smith - Feb 18th 2002 15:41:23

Already exists, here, in Freshmeat.

You may not like the API, you may not like the code and you may not like the current storage mechanism but, the concept is the correct answer to the problem. The rest can be fixed.

[reply] [top]

[»] While we're at it...
by Cory Sharp - Feb 18th 2002 14:34:58

Who's benefitting from having config files it /etc, man pages in /usr/man, libraries in /usr/lib, cache and locks and things in /var, and the program binary in /usr/bin?

I suppose originally those things made it easier for the system, not necessarily the user or the administator: have PATH check /usr/bin, have man check /usr/man, go look for shared libraries in /usr/lib, and so on.

Let's have a bit more than a config read/write library. Let's encapsulate a program and all of its files into a single system subdirectory. Installation and uninstallation of a program becomes easy. Complete backup and recovery of an application becomes easy. A clean separation between programs is better than an arbitrary intermingling.

The Windows Registry is successful because it's actually easier than managing a specific config file. This common interface allows us to do it one better: still an easy add/get/remove interface, but intead of storing the data globally in the registry (or semi-globally in /etc), it's local in that programs directory. That's both more sane and more clean.

And, of course, it can be extended so that user prefernces for an application can overlay the global config through a similar structure in the user's home directory. With a properly thought out API, programs can now have very easy access to both system and user configuration.

Let's try more than to just chase tails. Let's lead the pack.

[reply] [top]

[»] Re: While we're at it...
by Miles - Feb 18th 2002 15:43:47

A registry-like API doesn't mean we have to change the underlying structure. Keeping man data separate from configuration files is IMHO a good thing. As long as the locations are well defined (which they are in UNIX), the common API to which you refer can handle the disparate locations for you and present a common repository model to you.

[reply] [top]

---

[»] Re: While we're at it...
by Cory Sharp - Feb 18th 2002 16:58:28

Of course it doesn't mean the underlying structure has to change. I'm asserting that it should change. It's not exactly mixing man data with configuration files when both only belong to a single application. And I'm not ruling out putting those in further subdirs through a metaconfig if it's relevant for the application.

Just as many global variables start to pollute a C program, I think /etc pollutes the filesystem -- in the sense that every application is a little bit of everywhere in the filesystem. Encapsulation helps clean it up.

Aside from a sense of inertia, what reasons are there to keep /etc going? Easy grepping? With that simple, unified config file interface, it's easy to write either 1) an application that greps it for you, or 2) an application that dumps all configs to stdout for grepping with existing tools.

I believe the benefits of keeping a unique directory tree for each program significantly outweigh any minor benefits of /etc.

[reply] [top]

---

[»] Re: While we're at it...
by Jason D. Hildebrand - Feb 18th 2002 17:37:11


> Who's benefitting from having config files in /etc, man pages in /usr/man, libraries in /usr/lib, cache and locks and things in /var, and the program binary in /usr/bin?
I am. One way I benefit is by knowing that I don't have to backup my entire system; I can backup the configuration of my entire machine by taking a snapshot of the /etc directory. So it only costs me about 2 MB, instead of 8GB to back up everything I need to reconstruct my system.


> Let's have a bit more than a config read/write library. Let's encapsulate a program and all of its files into a single system subdirectory.
> Installation and uninstallation of a program becomes easy.
It already is easy. If it's not easy for you, try a different distribution (like debian, for example).


> Complete backup and recovery of an application becomes easy.
I already do this. I use dpkg-repack to re-create a package from its constituent files on my system, which means I can take a snapshot of a package and its current configuration.


> A clean separation between programs is better than an arbitrary intermingling.
Why? I find that consistent location of files by function
has advantages (as stated above) over lumping all application files together, especially when your packaging system takes care of installation/deinstallation. How would you propose to separate things like shared libraries "cleanly"?


[reply] [top]

---

[»] Re: While we're at it...
by Cory Sharp - Feb 18th 2002 18:06:47


> Why? I find that consistent location
> of files by function
> has advantages (as stated above) over
> lumping all application files together,
> especially when your packaging system
> takes care of
> installation/deinstallation. How would
> you propose to separate things like
> shared libraries "cleanly"?
Shared libraries don't create any issues. Cast ldconfig cast into this model. ldconfig lives in its own tree. It discovers shared libraries either by querying the config library or by direct report from the application with the shared library. The shared library is its own application, so all of its relevant man pages and configs are store in this unique-dir model under, as well.

This primary config application itself lives by its own rules, in its own unique subdir. And, if we're fitting this into the "registry" model, then that application is necessarily aware of the location of all applications and their config files (through preferably a discovery/report model like the bizarro-ldconfig, for robustness). From there, it is trivial to write an app that dumps the location of all config files to stdout for feeding into tar for backup.


[reply] [top]

---

[»] Re: While we're at it...
by Jason D. Hildebrand - Feb 18th 2002 18:27:27


> This primary config application itself
> lives by its own rules, in its own
> unique subdir. And, if we're fitting
> this into the "registry"
> model, then that application is
> necessarily aware of the location of all
> applications and their config files
> (through preferably a discovery/report
> model like the bizarro-ldconfig, for
> robustness). From there, it is trivial
> to write an app that dumps the location
> of all config files to stdout for
> feeding into tar for backup.
>

Fair enough, but why would I want to implement this when I can do tar -vzf etc.tgz /etc today?

What are the compelling reasons, in your mind, for wanting all application files to live together? What would that allow you to do that can't already be done?

[reply] [top]

---

[»] Re: While we're at it...
by Cory Sharp - Feb 18th 2002 20:34:07


> What are the compelling reasons, in
> your mind, for wanting all application
> files to live together? What would that
> allow you to do that can't already be
> done?
A reasonable request. In many ways, I want this for the same reasons you like the current method: ease of doing things By Hand. I suppose that's hard to define exactly, but something like "doing something By Hand means being able to exploit a variety existing tools to accomplish a task."

So, installing a package, completely uninstalling a package, relocating a package to a different directory or filesystem, being able to trivially inspect the resources (libs, man pages, binaries) that constitute a package -- these are things I'd like to do easily By Hand.

I can do some of those things with RPM, and perhaps from the sound of it, even more with the tools in Debian. But these tools and others (linuxconf) live more inside the system than on top of it. Which means I'm dependent on those tools to do the job, even when I otherwise wouldn't mind having other, simpler options. Abstacting to a lightweight layer to solve the problems curretly attacked by these heavyweight applications enhances robustness of the underlying structure and allows more versatility and variety in the front-end applications.

Now, I'm all in favor of tools that make things easier. But, I beleve those tools should supplement the system, not take it over. So, if we can exploit basic filesystem structure to accomplish more than these heavy management systems that push other tools out of the way, I think we should do it.


[reply] [top]

---

[»] Extremely Important
by Waldo Jaquith - Feb 18th 2002 13:48:04

Creating a unified configuration front end is perhaps
the single most important task remaining prior to
widespread Linux adoption. I suspect that we're
approaching a saturation point on acceptance of Linux
by Windows admins until we have a system of this
nature. Further, there are a lot of admins (mine
included) whose lives would be simplified tremendously
by this system.

Sure, I know how to appropriately modify Sendmail/
Apache/BIND/xinetd/etc., but the fact that everything is
so spread out is not at all helpful. If I want to add a
new domain to DNS, HTTPD, and my MTA, the process is
considerably more involved than it should be.

A unified XML-based data storage system (thus both
human and machine parseable) is clearly the solution
to this process, allowing any number of configuration
applications to take advantage of this common storage
system and simplify the process of Linux adoption as
well as make things easier for existing Linux
sysadmins.

[reply] [top]

[»] So where do we start?
by Morten Brix Pedersen - Feb 18th 2002 10:55:32

I like the article and the good-hearted purpose. But is anyone taking up the task?

I feel that this is "empty talk" if noone is going to take up the quest to start such project. A project to write documents about the configuration formats, create the APIs for different languages and to try getting this into LSB.

I would like to be a part of such project and to contribute with code and ideas, but I don't feel like I have the skills nor the time to lead it.

[reply] [top]

[»] Missing : Revision Control
by macc - Feb 18th 2002 10:16:06

What i miss with all config files is some kind of revision control.
to be able to
- back out a disastrous change affecting more
than one config file with a couple off keystrokes
- to have distinct config states
- to do temporary changes and return to your old config
- to merge in those new options that have been introduced
but not loose your current config
- whatever else your tortured mind comes up with

[reply] [top]

[»] Re: Missing : Revision Control
by Miles - Feb 18th 2002 15:30:45

I'd prefer that this was handled at the filesystem level: something akin to SnapFS, but it's development seems to have faltered. I believe the idea is sound however and solves more revision issues than just configuration files.

[reply] [top]

---

[»] Re: Missing : Revision Control
by Müller Miklós - Feb 18th 2002 16:02:02

Check RCSVI. Its a simple wrapper around vi that puts files under revision control. It's very simple and does not answer half of your needs but hey. The concept is there.

[reply] [top]

---

[»] We have a perfect language to handle this
by quinticent - Feb 18th 2002 10:12:07

Everything the author put forth can be handled by XML. Providing a Schema (I prefer RELAX-NG to XML Schema) for Linux configuration files to use as a standard will allow vi/emacs lovers to edit a config file by hand and still keep a file well formed for front end config tools. In fact XML goes a step further and allows for easy translation via XSLT. This makes automating building of frontends from the actual config files easy. Frontends would no longer need the amount of manpower currently required to keep it in sync with current config file formats. Think HTML frontends, textbased frontends and graphical GNOME and KDE frontends that drop right into the control panel when installed. An author would just have to write an XML skeleton for the config and any of these frontends would instantly work. Sounds like a good direction to go in.

--
J5

[reply] [top]

[»] config management on a PDA
by Cam - Feb 18th 2002 07:59:11

I am interested in configuration from a point of view of the Linux PDA. There is a real need for lightweight, centralised config management. Some of the requirements are:

* low memory usage
* one interface to the config, not duplicated effort (ie. one option setting library vs. every app having a custom written options page)
* persistence of options (obvious)
* buffer changes to prevent continual writes to disk/flash
* ability for apps to behave more like daemons in restarting and re-reading config files on the fly.

The best thing about doing this on a PDA is that it is a limited platform, so it is a feasible target to rewrite the config interface to every app and daemon on the platform.

There is a good page on config storage here:

http://www.cbbrowne.com/info/configdb.html

I used to have a link to a linux registry project that was widely dissed, I think for the wrong reasons. I hope they didn't give up. IMHO there are great gains in taking the responsibility for config file parsing, editing, restarting and so on away from the application programmer.

Some other comments on the article:

* application programmers should be able to 'layer' their option changing interface over the standard - eg. to provide a visual representation of page margins, line spacing, paragraph positioning etc. instead of the underlying numbers.
* internationalisation should be a separate issue. Config values are code. Static language dependent prompts should be internationalisable. But if a french guy wants to rename his 'Inbox' it is a matter of personal preference (config editing) not internationalisation. All we need is the ability to set default config settings depending on language preference.
* you do not mention management of config between different versions of applications. Imagine being able to: back up Abiword settings; update to latest version; revert to previous version (and settings)... all with minimal effort on the part of the application programmer.
* the trouble with maintaining a master copy in existing files is the potential for damage by a user.
* no mention of the ability to store the config centrally yet present it as a number of text files... think /proc

And I wish people would stop knee-jerking and shouting XML. That is great for interchange between systems but has little to offer as a config file repository. All the man-readablity of a binary database and all the bloat of text files. IMHO we need text files or a binary registry. Imagine a registry coded with unix values at heart, presented as a filesystem of text files: /conf a bit like /proc... I dream on!

-Cam

[reply] [top]

[»] GConf
by Christophe Saout - Feb 18th 2002 07:42:49

I always wondered, if we couldn't use GConf for this.

(as you know, GConf is a new backend configuration engine used in gnome).

GConf is modular, it uses plugins to store the configuration. The default plugins are xml files oder dbm files. The whole configuration on the frontend is shown like a big registry (like in windows), so the interface is easy to use and allows nice configuration applications to be written, but in contrary to windows, it doesn't write everything in one big file, but in separate files like tradiotional unix configuration files.

Now, if you write plugins to handle all the file formats in /etc, it should be possible to use GConf to handle this quit nicely.

Another point why GConf is great: The communication with GConf is done through a server process which keeps track of modifications. You can register callbacks if someone changes the configuration. So e.g. if someone changes the apache configuration file, you would immediately know it and send HUP to it. Or something like this.

You could even use it to do remote administration since it uses CORBA as communication engine, but there remain some security problems...

[reply] [top]

[»] Re: GConf
by Karl M. Hegbloom - Feb 18th 2002 13:01:48


> I always wondered, if we couldn't use
> GConf for this.
>
> (as you know, GConf is a new backend
> configuration engine used in gnome).
I think that this is exactly what "gconf" and "debconf" are designed for. (Perhaps "debconf" can use gconf? I'm not sure yet, since I've not spend much time studying this problem yet.) If "gconf" is not completely up to the task, then perhaps it is where development should be focused, to make it capable enough.


[reply] [top]

---

[»] An adaptable configuration system
by Laurent Pointal - Feb 18th 2002 07:37:20

For a project I develop at work, we have a similar problem: what to use for configuration files. In previous developments (under Windows but the problem remain the same), we used INI files, registry, registry mimic via directories and text files.
None was perfect, and the bad new is that each time our code depend on that configuration system.

So we develop a configuration front-end, using configuration resource URL, and entity name in that configuration resource. A function resolve the resource URL and the entity name and return an object which allows to access informations about the entity at the URL (after that informations are stored as paired name=value grouped by logical named sections).

Our code has not to know about configuration storage format and it has not to know about the protocol to access the information and the storage format, its processed by intermediate tools in the configuration library.

Currently we have only support for our text file format (see below), but i'm working on registry storage access (to include one of our new libraries into an older project - this library will need no code modification to use registry in place of text file).

Such a configuration tool allows you to store configuration data anywhere (maybe on remote computer, accessing it via LDAP or CORBA or SQL...), again its upon the configuration protocol support libraries to deal with that and to offer a standard interface.


Ex. of URL: fileconf:C:\confs\myconfig.txt
Ex. of URL: regconf://HKLM/Software/LURE/Disp
Ex. of our text configuration file
# This is a conf file
# Start an entity:
@Motor Bragg
# Start a section
[Unit]
unit=cm
longunit=centimeter
# Start another section
[target]
device=Motor2
protocol=CORBA
# Start another entity
@Motor Alpha
#... & co


A+

Laurent Pointal
CNRS - LURE

[reply] [top]

[»] Re: An adaptable configuration system
by Miles - Feb 18th 2002 14:38:11


> at the URL (after that informations are
> stored as paired name=value grouped by
> logical named sections).
In other words, standard Windows INI format. A format that Microsoft itself has abandoned for all of its programs because of its limitations.

1) It allows for only two levels of hierarchy (group and key/value pair)

2) Has no provisions for meta-data or structure enforcement
2a) because of this, a configuration tool would have to know about every program it configured in order to update as opposed to a unified method of defining what parameters are allowed and what types are allowed in that parameter.

3) Comments are not associated with the parameters to give context.
3a) Configuration tools would have to either assume that all comments above (or below or next to) a particular parameter were associated or none of them are and thus probable discontinuity between what the UI says the parameter does and what the config says that the parameter does.
3b) There is no distinction between a parameter comment (what the parameter does in all cases) and an instance comment (what the parameter does for your particular installation).

Addendum: Does your parser read and process Unicode characters properly? Can you assume that all programs who use this library will have no reason to store non-Latin1 values?

Now you can either try hacking these items into your config library/format or you can just bite the bullet and use XML.

[reply] [top]

---

[»] Further ideas for new approach
by Oliver Kiddle - Feb 18th 2002 06:19:25

I've thought about this before and considered starting a project. I used to do sysadmin on AIX and the AIX smitty tool is great. You have both a curses and GUI interface. One of the things I really liked was that by pressing F6, it would list which UNIX commands it was going to run to make the change. I could still do much of my administration with vi and from the command-line - much better than Linuxconf.

The article talks a lot obout configuration files but where possible external commands should be used - useradd/usermod etc, chkconfig for init levels, the print subsystem. For these things shell scripts could be stored and processed through m4 to cut them down to be as minimal as possible.

I also think XML is fairly suited to this task (though I've seen XML used in places where it shouldn't have been). We could start with an XML definition of all the configuration possibilities (though using the file-system for much of the division). XSLT could then process this down to make a platform dependant set of files. So an application could install its own XML definition files and the config tool would instantly be able to configure it. The XML might be a little like an HTML form with more info on processing. The config tool should definitely also allow different interfaces, especially including curses (text mode).

I'd definitely be interested in helping work on such a project so if anyone starts one, let me know.

[reply] [top]

[»] Kaos unix configuration
by Andrew McPherson - Feb 18th 2002 06:16:46

I'm developing a new Operating System, Kaos.
Kaos is a rewritten BSD with an exokernel, XML
filesystem, MIME database and encrypted filesharing/
IM.

I use XML as the main document satndard so I can tie
the filesystem to the search database to MIME to
filesharing and configuration files.

By using XML as a semantic web, a Grid operating
system can be installed, run and operated inside a web
browser running on your old OS before you startup
from the Kaos ramdisk loader.

This means configuration files are easy to edit, setup in
a web browser or use in a GUI program.
This also allows for SQL tables, java wizards or perl
scripts as plugins for even more simplicity.

Kaos will accept all existing BSD code and is likely to
accept some standard linux formats.

--
Kaos BSD operating system in development http:// neural.net.nz

[reply] [top]

[»] Re: Kaos unix configuration
by cfadam - Feb 18th 2002 14:18:25

OK, so where is the code? I checked out your site and saw nothing but a "wish list" of features you want to implement. Personally, I liked your idea, but if all you are going to do is spam this post with links to your vaporware OS, then take a hike.

The proof is in the code man. I also found it amusing that no part of your website may be stored in a computer per your copyright.. are you unaware of caching?

Best of luck in the future, but I still suggest you give back to the community from which you take. Your site mentions nothing about OSS, nor any plans to release your code under any open license. Maybe if you open-sourced your idea you would get some like-minded developers to help, but instead you have a 6yr old site dedicated to an OS that probably won't ever make it past the drawing board.

[reply] [top]

---

[»] In With the Old, In With the New
by Kathryn Andersen - Feb 17th 2002 20:10:05

I agree with most of the things said in the article -- an easier common configuration is needed. Ignore the flamers who want to stay in their 3l33t hell -- I use vi(m) with the best of them, but I'd still rather have a tool which helped me pick options quicker and still enable me to edit the config file later by hand if I wanted to.

Two main partial solutions have been proposed, Webmin and Gconf. Webmin is cool, I use it myself, but I agree that it has limitations, mainly the web-only interface, and the lack of including of the helpful comments. At least it does allow you to go back and edit the original config files, rather than mangling them beyond recall, as other tools have done in the past. Gconf may be cool, but it has even more of a hurdle -- correct me if I'm wrong, but I gather it requires the application to be rewritten to use Gconf, it won't work with existing text configuration files.

If one really wants a common configuration interface one must make it easy for the application writer to write the "meta config" files for it -- because the harder it is, the more likely that the application writer simply will not bother, and there goes your dream of the application writer keeping the interface up to date for you.

Whatever the system is, it shouldn't require the application writer to rewrite their application in order to use it -- though they may if they want to (the idea of being able to call the config engine from an application's File->Preferences menu is cool, but not if it doesn't have a File->Preferences menu!). But, realistically speaking, hands up how many people think that the author of Sendmail will rewrite it to use Gconf? Oh, really? Well, I have this bridge I can sell you...

The point is, that the interface is not required to be an interface with the program itself, but with the program's configuration files. The idea of backend and frontend plugins is a good one, so long as whatever does the translation (to XML, maybe) does preserve all of the original configuration file's information -- or we'll get back to the trash-the-config-file-hell we want to avoid with a general configuration tool. Plugins allow flexibility and, hopefully, the application of more volunteers to help make the tool more useful. So what if some particular config file style is only used by one particular application (say, Sendmail...) -- if somebody wants to write a plugin for it, that's another blow for democ- well, a blow for diversity, anyway.

An alternative to having the application writer write the metafile would be to have volunteers write them for particular applications if the application writer in question won't do so. After all, we can't assume that every application writer will do so, and presumably it shouldn't be harder to get volunteers for such an effort than it is to get volunteers to make Webmin modules (or Debian packages, for that matter). (Though, how hard is it to write a Webmin module? Not trivial, I expect.)

Another reason such a configuration tool needs to be able to read original existing configuration file formats is that users won't be too pleased at losing all their customisations the first time they use the tool because their old config files are no longer used.

(Another problem I have with Gconf is really just a bleeding edge problem; I tried to get the most recent version of Galeon running, and it wouldn't because I didn't have the latest version of Gconf. But that's more of a Galeon problem than a Gconf problem and I shouldn't blame Gconf for it...)

(And while I'm going for asides, the person who was complaining about how hard it is to get at the documentation, since some is manpages, some HTML, some info, some Pod and so on -- Webmin has a really useful module for System Documentation, which will search all of the above documentation types for you, and give you access to all the documents it finds. So it doesn't matter if it's a man page or a HTML file, or a plain text README file, you can read it.)

Enough asides. But I think if I go on I'll start repeating myself....

--
Say it with flowers -- send a Triffid.

[reply] [top]

[»] vi rulzZz..
by Stonefly - Feb 16th 2002 16:39:24

well, you really think it is possible to create a better configuration tool than vi??????

all-in-one configuration tools suck, because 2 seconds after publishing them, someone releases a new version of his/her program with millions of new features and your super trooper linuxconf/yast2/orhoweveruwannacallit is out of date.

one last question: how can i use a "graphical point-and-click" configuration tool on my server without a mouse/X-server???

regards,

Stonefly

[reply] [top]

[»] Re: vi rulzZz..
by Buchan Milne - Feb 16th 2002 17:02:06


> well, you really think it is possible to
> create a better configuration tool than
> vi??????
Not necessarily. Maybe we can make vi display help on the configuration entry you are using. Maybe not all people care to learn vi.


> all-in-one configuration tools suck,
> because 2 seconds after publishing them,
> someone releases a new version of
> his/her program with millions of new
> features and your super trooper
> linuxconf/yast2/orhoweveruwannacallit is
> out of date.
That is one of the specific things a tool like this would address. By having some sort of meta-configuration information (shema files) which are maintained by the project the files configure. Thus, they should never be out of date.


> one last question: how can i use a
> &quot;graphical point-and-click&quot;

> configuration tool on my server without
> a mouse/X-server???
A graphical interface would only be one frontend to a really good backend. Tell me you wouldn't like to be able to do:

# config http add_virtual_host my.domain.com /var/www/mydomain.com

[reply] [top]

---

[»] Re: vi rulzZz..
by Stonefly - Feb 16th 2002 17:22:37


>
> % well, you really think it is
> possible to
> % create a better configuration tool
> than
> % vi??????
>
>
> Not necessarily. Maybe we can make vi
> display help on the configuration entry
> you are using. Maybe not all people care
> to learn vi.
> u wanna make vi doing what?
are u sure u know what a console based text editor is?

>
> % all-in-one configuration tools
> suck,
> % because 2 seconds after publishing
> them,
> % someone releases a new version of
> % his/her program with millions of
> new
> % features and your super trooper
> %
> linuxconf/yast2/orhoweveruwannacallit
> is
> % out of date.
>
>
> That is one of the specific things a
> tool like this would address. By having
> some sort of meta-configuration
> information (shema files) which are
> maintained by the project the files
> configure. Thus, they should never be
> out of date.
> oke, one point for you

>
> % one last question: how can i use a
> % &amp;quot;graphical
> point-and-click&amp;quot;
> % configuration tool on my server
> without
> % a mouse/X-server???
>
>
> A graphical interface would only be
> one frontend to a really good backend.
> Tell me you wouldn't like to be able to
> do:
>
> # config http add_virtual_host
> my.domain.com /var/www/mydomain.com i would like to be able to change the mtu settings on a windows machine with a text editor

[reply] [top]

---

[»] Re: vi rulzZz..
by Buchan Milne - Feb 17th 2002 07:48:40


> u wanna make vi doing what?
> are u sure u know what a console based
> text editor is?
You could have vi display the hints for the configuration option in another buffer (like the way the help works). OK, maybe emacs would be better for this. But yes, I know what a console editor is, I currently use vim-enhanced about 15% of the time I use a computer.


> % % one last question: how can i use
> a
> % % &amp;amp;quot;graphical
> % point-and-click&amp;amp;quot;
> % % configuration tool on my server
> % without
> % % a mouse/X-server???
> %
> %
> % A graphical interface would only
> be
> % one frontend to a really good
> backend.
> % Tell me you wouldn't like to be able
> to
> % do:
> %
> % # config http add_virtual_host
> % my.domain.com
> /var/www/mydomain.com
Of course, I forgot to mention that this would make the tool scriptable, so you could do:

# for i in my yours hers everyone;do
config http add_virtual_host $i.domain.com /var/www/$i.domaim.com;
done

Wouldn't that be cool?


> i would like to be able to change the
> mtu settings on a windows machine with a
> text editor
C:\>regedit /s mtu.reg
(you use the text editor to create mtu.reg of course)

But then you run into exactly the same problem. Which registry keys do you change? What value do you need to put into the key?

Maybe we could make a backend for you that writes a .reg file for you, and then installs it on the remote machine for you. If you are working locally, it's even easier.


[reply] [top]

---

[»] Re: vi rulzZz..
by Wilbert Berendsen - Mar 15th 2002 02:16:54


> You could have vi display the hints
> for the configuration option in another
> buffer (like the way the help works). Then why not just commented config files. I find that very intuïtive. Explanation can just be added in the file as comments. You can enter different config options, and easily uncomment the desired one. You can write comments yourself, to remember why such-and-such config change was done.

[reply] [top]

---

[»] And what about using "Brain"
by Robert Trebula - Feb 16th 2002 12:53:48

Yes, I mean the thing stuffed inside your head.

Why do you need single configuration interface to
all things that can be configured? Are you too lazy
to learn how to configure each single one by one?
Are you too lame?

For real, how many ways to configure things do you
know? How many formats there exist? How many do
*you* need to know? Let's count: you need to set up
mail server, http server, samba, mail client, database
server, window manager... How many have you
counted to?

Can't you just learn how to configure these? I,
personally, like learning stuff. Maybe you could give
learning a try. I promise you'll learn much more than
just configuration. You'll get somehow deeper look
inside the thing you are configuring, how does it
work. When you know what and why you are
configuring, you'll definitely configure it better and it
will work better for you.

The best example I gould give you are "commented
configuration files". Look at /etc/postfix/main.cf or
/usr/local/apache/conf/httpd.conf e.g. Have you
forgotten to read text? Have you forgotten how to
launch a text editor and change some values in a
file? What do you learn by clicking on checkboxes?

Ah, you were talking about "newbie users". I believe
that the best one can do to have his product easily
unterstandable and configurable for newbies is to set
up a fully functioning configuration 'out of the box'
and keep the configuration files commented and
simple to understand.

Robert

P.S. and sorry if this sounded like flame. It wasn't
meant so.

[reply] [top]

[»] Re: And what about using "Brain"
by Sean Middleditch - Feb 16th 2002 13:50:54

Um, what about people that *don't* want to learn, or don't have the time? I know how to configure 90% of the average Linux server applications without using any documentation, but it certainly takes a long while. It's a hell of a lot faster to point and click a few times, and be done with it.

Not to mention that there *are* newbies to the system that don't want to learn at all. They just want to get the task done without spending a week learning how to setup a webserver.

I certainly would hope your only reason to not making configuration simpler is your ego and trying to feel l337't than everyone else.


> Yes, I mean the thing stuffed inside
> your head.
>
> Why do you need single configuration
> interface to
> all things that can be configured? Are
> you too lazy
> to learn how to configure each single
> one by one?
> Are you too lame?
>
> For real, how many ways to configure
> things do you
> know? How many formats there exist?
> How many do
> *you* need to know? Let's count: you
> need to set up
> mail server, http server, samba, mail
> client, database
> server, window manager... How many
> have you
> counted to?
>
> Can't you just learn how to configure
> these? I,
> personally, like learning stuff. Maybe
> you could give
> learning a try. I promise you'll learn
> much more than
> just configuration. You'll get somehow
> deeper look
> inside the thing you are configuring,
> how does it
> work. When you know what and why you
> are
> configuring, you'll definitely
> configure it better and it
> will work better for you.
>
> The best example I gould give you are
> "commented
> configuration files". Look at
> /etc/postfix/main.cf or
> /usr/local/apache/conf/httpd.conf e.g.
> Have you
> forgotten to read text? Have you
> forgotten how to
> launch a text editor and change some
> values in a
> file? What do you learn by clicking on
> checkboxes?
>
> Ah, you were talking about
> "newbie users". I believe
> that the best one can do to have his
> product easily
> unterstandable and configurable for
> newbies is to set
> up a fully functioning configuration
> 'out of the box'
> and keep the configuration files
> commented and
> simple to understand.
>
> Robert
>
> P.S. and sorry if this sounded like
> flame. It wasn't
> meant so.
>
>


--
-- What?

[reply] [top]

---

[»] Re: And what about using "Brain"
by Travis Gerspacher - Feb 16th 2002 16:55:30


> Not to mention that there *are*
> newbies to the system that don't want to
> learn at all. They just want to get the
> task done without spending a week
> learning how to setup a webserver.
Why are we worried about newbies who don't want to learn at all. Making things easier is one thing but totally revamping everything that people have worked on in order to support people who aren't willing to learn.

If you can't read the documentation and manage the softwares configuration files, then maybe you shouldn't be running the software.

I personally think that some packages such as apache, samba and many others have quite understandable config files. Although they are completely different, understanding the configuration is part of the learning process. I say tough shit, if you aren't willing to learn.

[reply] [top]

---

[»] Re: And what about using "Brai